As part of 2014 Next Wave Festival at the Brooklyn Academy of Music (BAM) and their series of events centered around Nonesuch Records, storyteller/musician Laurie Anderson has collaborated with the Kronos Quartet to create Landfall, a piece that is based on Anderson’s experience during Hurricane Sandy. In this evocative meditation on transience, texts and music are enhanced by dense projected texts, triggered musically via software developed for the work, extending Anderson’s use of multimedia. Live Design chats with digital artist Liubo Borissov, the programmer behind the projections, who used custom-built Erst software.
Live Design: Please describe the Erst software and how it allows for the musical triggering of the projected bits of language and code.
Liubo Borissov: Erst is a custom-built performance system designed for Landfall. The goal was to make an audio-visual instrument than can be performed by an operator as a part of the ensemble. On the simplest level it can work as a cue system, which can be triggered by hand while following a score. On another level, it does real-time audio analysis that can detect musical events such as the onset of a note, a specific pitch or pitch change, or a particular articulation produced by the performer. These events can be mapped onto media events, such as showing a line of text, a symbol, an image, a color, or a type of motion. Ultimately, it is a hybrid system, partially controlled by the operator and partially automatically responding according to predefined rules.
LD: Who did the content creation for the images and what is the playback system —is that Erst as well?
LB: Erst is both a content creation and a playback system. It has a primitive text editor that allows the artist to define and edit visual cues and their relationship to musical events. What is unusual about the editor is that it enables attribute modification and statistical control on an individual character level—so font, color, and size can change within the same word but how the change occurs is determined by adjustable probabilities. One of the goals during early development was to not make just a visual backdrop to the piece but to try to obscure meaning, hopefully pulling the audience in and leaving more room for imagination and association. This is the reason why I am reluctant to reveal too much about the inner workings of the system, but one approach was to start with a text from Laurie and use different alphabets, e.g. from little known or lost languages or a collection of road signs or scientific symbols. We would then allow chance to determine how much the original text would be encoded through these ciphers. Aside from the design of the software system, I collaborated with Laurie on creating the visual alphabets and codes, but the system was built in a way that she could realize the actual content, words, stories, images, within Erst herself as the project traveled.
LD: how do the images and music go hand-in-hand?
LB: The system allows for making content decisions during iterations of the development process and recording them as part of the score. In practice, we rely on the proclivity of the brain to look for and discover patterns. For example, one of the early experiments was reading one text out loud and having the system respond by following the voice and displaying the words of another. The experience of trying to parse two different dissociated texts through two different senses is odd. The brain grasps for meaning and association where perhaps there was originally none, but the process itself fills that void. Such experiments informed the final choices for juxtapositions of sound, text, and media, which were determined largely by the narrative elements of the piece.
LD: What are the projection surfaces in Landfall?
LB: Since everything is controlled by a single computer, there is a lot of flexibility about the output. The projection surfaces may change depending on the venue or choices by the lighting and stage designer.
LD: What would you say is the emotional reaction intended by the images?
LB: My personal take—and drive—during the design was to try not to explain too much. I feel that in an age of sensory and information overload, erasing or concealing can have a stronger impact than overwhelming with content. The music and stories alone already evoke very strong emotions— my goal for the visual system was to create an environment in which they could inhabit.
LD: What was the largest technical challenge for you in the creation of this piece?
LB: The largest technical challenge when making art software is that it inevitably has to be an iterative process and most plans go out the window as the piece evolves and develops. It flies against every rule of proper engineering design, and in many ways one has to be willing to improvise with code. This is exactly why I love doing it.