Out with the Old, In with the Ethernet

As a lighting engineer at the television network QVC, I face, on a daily basis, unique scenarios that challenge me to stay on the cutting edge of technology. Two years ago, I was asked to help with the transition from a DMX network to a new, powerful Ethernet-based system. Since then, we have experienced many trials and tribulations, because of the system's size and because of the nature of our business-but we have in place a new Ethernet system that we continue to enhance.

QVC is the world's largest electronic retailer and the second largest television network in the United States. It broadcasts live 24 hours a day, seven days a week, 364 days a year to more than 85 million homes in the US, and to millions more in England, Germany, and Japan. QVC's broadcast lighting department consists of seven people: one lighting supervisor, two lighting engineers, two LDs, and two grips. It's a small number, considering they must handle nearly 40,000 sq. ft. of studio space in eight studio locations, lighting nearly 350 new sets a year. Currently, QVC has in stock roughly 1,000 conventional and 80 automated fixtures.

We had to choose Ethernet for this new system because, under our previous setup, we were out of channels, we had no room to grow, and yet we still were introducing more and more automated fixtures into the mix. Ethernet offered us an incredible amount of expansion, flexibility, and reliability. However, it was crucial to remove the old system and replace it with a new Ethernet network as seamlessly as possible.

Before the changeover, we had a standard two-universe DMX512 system was controlled by six Electronics Diversified Enact consoles in different locations, EDI Copy Cat units, and High End Systems LCD controllers. The signal flow in the Main Studio was complicated; we had only three Enact two-universe consoles running through DMX mergers and controlling 1,000 dimmers and analog-triggering multiple-LCD controllers running 22 automated fixtures and 12 scrollers, and triggering two huge roll drops. The control lines ran upstairs to our main dimmer room and through splitters to control our 12 dimmer racks, landing in multiple studio and control-room locations for patch points. We backed up the consoles like everyone else, walking around to each location and backing each unit up to a floppy disk. The system was in place for five years and was reliable — but we outgrew it.

Originally, we decided to create a skeleton Ethernet network — our backbone — that would be separate from the building's telecommunications and broadcasting systems. This system would offer us total redundancy of our lines, switches, and consoles. Since lighting supervisor Doug Rae and lighting engineer Doug Franz already had a working topology of the system, it made our console needs very specific, and we needed precise functionality. During the planning stages, we broke the system up into three phases: Phase One included bringing online the Annex (our theatre), placing the first round of purchases, and starting the network connectivity testing process. Phase Two consisted of bringing online our Main Studio, the second round of equipment purchases, and network testing. Phase Three took in Stage 3, the Film and Video studio, three post studios, the Show Kitchen, the digital lab, our office, and the respective dimmers.

Of course, choosing a console is the most important part of any system, let alone an Ethernet network (We planned for 10 consoles). Consoles today have come a long way; however, it's very rare to have one that you can rely on 100%. Before we started shopping, Doug Rae, Doug Franz, and I came up with a list of specific requirements:

  1. A console that could have multiple units, of the same manufacturer, networked together, all running independently of each other;

  2. The ability to control any console, from any location in our network (a flawless master-master relationship);

  3. The ability to backup any console, in any location, without disrupting the live show, separate from the network;

  4. A way to access a certain area of the studio and not affect any other part of the system. This was a key element because we deal with 1,000 dimmers in the Main Studio;

  5. A console that could take over seamlessly, with no interruption, if any other unit were to fail on the network;

  6. A console that could periodically save information locally in case the server crashed;

  7. The ability to have multiple wireless handheld units performing most or all of the functions of the console. This would aid us in troubleshooting, maintenance, and bad line-of-site areas;

  8. A 3D visualization program that could be integrated with all consoles in specific locations. This would help us in pre-lighting larger-than-normal shows. The 3D program would also have to be able to import AutoCAD and 3D Studio Max drawings;

  9. A network dial-up connection, so that engineers from the lighting team, or the manufacturer, could access it directly for troubleshooting.

As always, other items were taken into consideration, including user-friendly qualities (because we would have floor managers, set dressers, scenic people, and interns using it as well), stability, processing speed, expansion, technical support, and so on.

THE RESEARCH

We reached out to many console manufacturers, asking for demos, finding out how fast they could implement what we needed. Surprisingly many of them couldn't, and their consoles didn't have the learning curve we needed. So, having used the MA Lighting grandMA console on previous shows, I contacted Bob Gordon at A.C.T Lighting (the US distributor of the grandMA line) and spoke with him about our goals. Two days later, a grandMA and grandMA Light were sitting in my shop. Shortly after that, Mike Falconer called from A.C.T, and we discussed what we were trying to accomplish and how far this console would be able to take us. At this point, the grandMA could not give us what we wanted; however, the console did offer an amazing user interface, making it quite possibly one of the easiest and best consoles to learn. It also offered a platform that was extremely stable, with amazing processing power. By far, the grandMA was our number-one choice.

Thus we needed to Beta-test our ideas with our new set of grandMAs. The testing was spread over three distinct time frames. First, we were to receive the consoles and determine whether or not they would work for us. Next would be the release of the new software (4.0). The third period would be an ongoing process, to happen every time there was a software update or when we would expand our system.

Our Beta-test area was the Annex. It is basically a small black-box theatre with a couple of pantographs, 20 automated fixtures and about 200 dimmers; it was the least used of our studio areas, which made it the safest. Understand that, working in a live studio, we must be prepared to jump into any studio at any time. This means that all of our consoles stay on, as well as our automated fixtures — we never power them down, unless we're performing maintenance. This created a perfect testing ground-and also an uncommon stress on the consoles. I placed one grandMA on the studio floor — this would be our floor location — and one grandMA Light in our booth for running the shows. They were connected via Ethernet and a powered switch, and I ran them in a tracking backup mode. Whenever I switched control from the floor to the booth (or vice-versa), I had to disconnect them from the tracking backup state and reverse the process, which caused a reload of the current show as well. As tedious as this sounds — and it was — the information that I was able to record and feed back to Mike Falconer was priceless — it helped pave the way to the new networking version of the console.

From the aspect of functionality and reliability, the grandMA was, in terms of traditional console usage, perfect; however, it still didn't meet our network requirements. Clearly, we needed a Master-Master control scenario. Mike and I had many phone conversations about this. The time and persistence paid off when Bob and Mike visited QVC with a special guest, Michael Adenau, founder of MA Lighting, to walk through our studio complex and hear our concerns in person. We met with them for a few days; Michael Adenau made some calls to Germany, we said our goodbyes, and were given a solid handshake then said; they'd try to have a Beta for us by LDI (2001). As MA and A.C.T worked on the nine bullet points that we had given them, they helped us every day, in every situation, making our Beta site a temporary semi-success. MA released a quick version of software for us, Version 3.4, which headed off most of our small network stability issues for the time being.

TRANSITION AND IMPLEMENTATION

The grandMA console from MA Lighting has been attracting professional programmers and newcomers because of its hardware environment, including three-color touch screens, free visualization software, motorized faders, complete configurability, hard drive, and Ethernet, among other things. But it was with the advent of Version 4 of the operating software that the console really delivered on “next-generation” capabilities that had not been seen before in a moving-light-capable console.

Top of the list of next-generation features on the grandMA (even above PDA remote control and matrix programming), is multi-user programming. This is where two or more consoles can be connected together by Ethernet, and act as one, but with multiple programmers. All the grandMAs in the “session” have access to all of the fixtures connected to the console and all the cues. This could cause problems, as multiple programmers try to control the same lights or program different cues at the same time.

What makes a system like this workable is a concept called Worlds. A World is a group of fixtures or fixture parameters. When a console is in a World it acts as if those are the only fixtures connected to it. The displays only show those fixtures and, when storing, editing, or deleting, only the fixtures or fixture parameters included in that World are affected. Because changing Worlds is as easy as selecting a group of fixtures, the Worlds system brings flexibility and speed of operation to a multi-user enviroment. Even non-multi-user shows have started to benefit from Worlds as they allow for the sectioning of the consoles allowing programmers to work on several things at once without having to constantly clear or record.

That takes care of programming, but what about playback? There are two options in this multi-user scenario: “In-sync” and “Out-of-sync.” When a grandMA is “in-sync,” pressing Go on a playback will press Go on any other grandMA in the session that is “in-sync.” If a console is “out-of-sync,” pressing Go on a playback will not affect any other console and Go on another console will not affect it.

The bottom line is that all of this is at the press of a button (or touch of a touch screen if you prefer) where connected consoles, which share information, can act completely independently or act in concert.

Once we completed our research and Beta-testing, it was time to plan the implementation of the system. It had to happen quickly, but we didn't want to compromise the design or quality of the system (Altogether, the project will take about four years and we anticipate completion in 2004). Having seen Beta versions of MA Lighting's grandMA Version 4.0 software at LDI2002, we knew we'd be challenged with the software's learning curve and possible issues with our system turnover. The team at MA Lighting hit every bullet point on our list, except for the network dial-up connection, which was not a concern at the moment — and came up with an incredible networking plan. Now we were able to split each area into several sessions and have redundant backups. For example, we have a Main Studio session with three grandMA consoles in it, a Stage 3 with one grandMA console in it, and an Annex Session with a grandMA and a grandMA Light in it. This way, we could separate the different areas of our broadcasting locations, and could still leave any session, then join another session on our network, without shutting off the lights in any area.

As we started to receive our new consoles — and to update our older models to Version 4.0 — we tested them in our shop for about two weeks, then placed them in their designated areas. The Annex was completed with two consoles, two Artistic Licence Net Link O/Ps and a 3Comm Wireless Access point. Next, we placed consoles in the Stage 3 area and the Main Studio. The latter would be the most difficult part of the project. We placed three consoles in specific areas that would give our LDs the best sightline while not hindering the other studio operations, keeping in mind that any console would be able to control any area from any location at any given point in time. After all the consoles were placed, we laid down our own temporary network. This was a very delicate process because we had to wait until the live show was out of the area so we could make the necessary changes to the consoles and the network. I placed a hub in one of our broadcast engineering racks in the Main Studio, and from that point we connected three consoles, four Artistic Licence Net Link O/Ps and one 3Comm Wireless Access Point. Our main dimmer racks still use the old DMX lines, but are plugged into one of the Net Link O/P nodes from the Main Studio. This will be implemented when the main cable installation is completed, hopefully within the next two months. With our Main Studio session, our Annex Session, and our Stage 3 Session, this makes up our current lighting network system. When we need to troubleshoot our system, we use Artistic Licence's Net View and DMX Workshop.

END OF THE DAY

So, at the end of the day we have an incredible lighting system that allows us to expand in many different directions with confidence in our work. Now that we're able to sleep once again, we can take on the new projects that appear so frequently on the horizon. As lighting engineers, designers, and programmers, we must always be prepared to push the system in ways that were never intended. Whether it's a set that needs an enormous amount of fixtures or a fully automated lighting system (fixtures that track horizontally and vertically), we now have a system in place that can take us to that next level, which is whatever we can imagine.