Customized Video on the Fly for Concert Tour
 Photo: Lewis Lee. |
When legendary rock band Rush hit the road last year, the group did some experimenting with interactive 3D video along the way. During its 40-city Vapor Trails tour, the band used Derivative Inc.'s TouchTools visual performance technology to offer audiences a presentation designed to hearken back to the light shows of the 1960s, but orchestrated using a revolutionary new technique that has many potential implications for the touring concert world, as well as theatrical shows, museums, installations, and corporate events.
“Rush has always used visuals, and I'm a big fan of visuals as well,” says Howard Ungerleider of Canada's Production Design International, a veteran Rush tour manager who served as lighting designer and show director for the Vapor Trails tour. “TouchTools is leading-edge technology that has never been used this way before.”
Originally designed for creators of interactive art, Derivative Inc.'s Touch technology is based on Side Effects' Houdini 3D animation technology, enabling digital artists to author, perform, and share expressive, interactive 3D visuals from a PC platform. The relationship with Houdini is hardly an accident; TouchTools is the brainchild of Derivative president/founder Greg Hermanovic, the programmer who co-founded (with Kim Davidson) Side Effects Software in 1987.
“A couple of years ago, I saw the writing on the wall in terms of what realtime could be,” explains Hermanovic. “I knew laptop computers would be fast enough to do very good, realistic live 3D and that sooner or later, it would be in the hands of everyone, thanks to realtime graphics processing from nVidia and ATI.”
Started as a spin-off from Side Effects in 2000, Derivative retooled Houdini and ended up with a suite of tools for realtime animation performance. At the heart of the system is the Synth, or visual synthesizer, an image or series of images that can be played with the touch of a key, as if they were notes on a musical synthesizer. TouchDesigner is the full content-editing tool for creating and modifying Touch Synths, capable of modeling, texturing, and lighting objects in 3D, construction of control panels, and creation of motions and behaviors. After creation, TouchSynths are played-back on either the company's TouchPlayer (a free download) or the more professional-level TouchMixer, which offers MIDI I/O and more infrastructure for recording gestures, editing them, and saving the results. (For more detailed info on TouchTools, visit www.derivativeinc.com.)
 Two Dell Inspiron 8200 notebook PCs, with nVidia GeForce4 Go graphic processors, helped drive effects. Photo: Farah Yusuf. |
After hearing about Derivative, Geddy Lee, Rush's lead singer, and Ungerleider took a look at some early versions of Synths and promptly declared themselves in love with the concept. What they particularly liked was the Touch ability to improvise and reshape the visuals at will and in realtime, permitting the band to make the visual presentation different every night.
“I find that the Derivative's Touch tools ability to change images live in response to the music is pretty amazing,” says Ungerleider. He adds that the “Vapor Trails” tour also featured a live-action video camera, directed by David Davidian, and a High End Systems' Catalyst controller, which allowed him to trigger all visuals from the lighting console as a lighting cue.
Working with the Derivative team, Ungerleider planned and created various Synths, along with the control interface, for 10 weeks prior to the tour going on the road. One crucial element was finding an appropriate VJ, or visualist, who would be responsible for controlling the Synths live every show on the tour, responding to changes in tempo or mood.
“[The system's effectiveness] depends 100% on the talent of the VJ,” Ungerleider says. “Derivative's Touch Tools are very innovative because there are so many ways to manipulate the imagery. The VJ working with the programmers creating the Synths is integral, as well as finding a VJ who has the talent to perform well every night with these images.”
As it happened, James Ellis, a partner [with Dwight Rider] in Secret Sauce, a realtime music animation performance troupe based in Santa Clarita, Calif., called Hermanovic while the Rush team was busy creating the Synths prior to the tour. An early adopter of TouchTools and a former Cal Arts student in experimental animation, Ellis earned the VJ gig by proving to be both proficient in the TouchTools technology and a major Rush Fan.
 To accommodate widescreen projection, the Rush Synths were built with an image resolution of 1200x450 pixels. Photo: Todd Kaplan |
By the time Ellis got to Toronto for pre-rehearsals, a team of four animators had already been hard at work creating the design elements, so Ellis collaborated with them on performance and interface elements. On the road, the Touch kit included two Dell Inspiron 8200 notebook PCs, featuring nVidia GeForce4 Go graphic processors. Both notebook PCs were also loaded with the TouchMixer (version 009) and 12 TouchSynths. Ellis used the TouchMixer to control the movement of 3D characters, shapes, and animation lighting. Wired to each Touch notebook were a 25-key musical keyboard and the CM Labs MotorMix controller, a servo-driven, slider-button unit, which in tight integration with Touch software, kept the sliders in perfect synch with the software's onscreen sliders. This enabled Ellis to jump to preset cues and sequences, watch the controls move automatically, and then take over and improvise as he saw fit.
The two laptops were a prerequisite for quickly switching between Synths and songs. For backup and reference purposes, the entire visual performance was also recorded to DVD, which in an emergency, could have been used to play back the Synths. “This would only happen if a laptop were to completely break down during a show, and even then, it would be a temporary measure since a backup laptop can be swapped in about five minutes,” notes Ellis.
Once the system was up and running, and it came time for the actual live performance with the tour's 12 songs, success was more a matter of talent than technical proficiency, according to Ellis.
“The performance itself isn't that difficult,” explains Ellis. “It is very intuitive. You might say it's like playing the piano — it's not that difficult to physically play, but to get a real performance out of it takes time.”
 Effects for Rush’s "Vapor Trails" tour were designed to recall the rock light shows of the 1960s. Photo: Todd Kaplan |
Hermanovic notes that Touch visuals are ideal for live music performances. Although Touch animations can be very structured to precisely match cues and particular sections of the music, they also allow for improvisation within that controlled structure, for visual variations within the show from night to night, unlike edited video.
During the Rush concerts, the Touch Synths were displayed on a Sony LED screen behind the band that measured 40'×15', with an aspect ratio of 8:3. To accommodate widescreen projection, the Rush Synths were built with an image resolution of 1200×450 pixels on the Dell laptops. The images were then fed to a Folsom ViewMax scan converter, where they were cropped and squeezed into a conventional composite video signal. The video image was then stretched back to 8:3 when broadcast on the LED screen.
On tour, Ellis often divided songs into 16 parts and jumped to those sections using the MotorMix's preset buttons. Even with this many possible triggers, he needed to add a second MIDI device — a MidiMan Oxygen 8, 25-key musical keyboard — for certain animation sequences, such as the camera shakes and fluid character movement that accompanied the song “Leave That Thing Alone.” Ellis says that the ability to control the images with a MIDI device represents the “simplicity” of the approach.
“In our opinion, nothing beats the ergonomically refined feel of striking real keys on an actual musical keyboard,” says Ellis. “Performing with it isn't that difficult — the way it's constructed, you have some real-life controls over the virtual sliders.”
 During the concerts, the Touch Synths were displayed on a Sony LED screen behind the band that measured 40’x15’, with an aspect ratio of 8:3. Photo: Farah Yusuf |
The Rush tour ended up as not only a trial by fire for Derivative's TouchTools, but also as an ideal way to show off the technology in 40 North American cities. Hermanovic's goal now is to build a real community of talented VJs qualified to both create and perform Synth Labels over time.
“It's almost like you're building an application,” notes Hermanovic. “It's a bit analogous to Flash files, but Touch goes further. It treats the end-user as a performer, not a passive viewer.”
Debra Kaufman is a writer/consultant who has been covering the entertainment industry for 14 years. She can be reached at [email protected]