Immersive's projects range from superyachts to trade shows, the giant dome in Philadelphia's Comcast Center to musician Jason Mraz's world tour and everything in between. One of the first agencies to pioneer immersive storytelling, founder, chief creative officer, and CEO, John Munro, used his background in art and design to combine traditional forms of expression with cutting edge technology. Managing director and executive creative director, Tommy Lexen, joined the company after 15 years producing and directing large-scale events including theatre, VR, exhibitions and trade shows.
Live Design talked to Munro and Lexen about the future of immersive storytelling and what today's designers should be preparing for tomorrow.
Live Design: Immersive can mean a lot of things – from physical gags on a stationary audience like the seats rigged to spray water back in the early 90s at Universal Studios, to walk-through 360-degree projections on shows like the traveling Van Gogh exhibit. How would you define immersive?
John Munro: When I founded the company back in 2004, I intended to connect architecture and storytelling experiences. My background is in spatial design, design theory and fine art, so I wanted to create a company that could link these aspects together to create a new kind of interactive experience. I called the company Immersive when it was not a buzzword; it was more a statement of intention.
Personally, I see the definition of Immersive as a way to transport our audiences into a shared world of inspiration and wonder.
Back in those days, we had to create our own hardware and software to be able to project onto bespoke structures to enable immersive environments, but for us it has never been purely about technology.
Immersive spatial storytelling is a unique form of art and design, and it can bring people together in a way that no other medium has been able to so far. It combines all the best elements of film, music, art, design and architecture as its content and canvas.
LD: For stage designers used to a fourth wall, how do they adapt to immersive entertainment? Is there a mindset that needs to be adopted rather than just considering lighting angles from multiple directions?
JM: Designers do need to adopt a new mindset about how their audience experiences these works. To direct, you must think spatially, and I think you need some architectural training. And to be able to direct animation and film, you need a deep understanding of camerawork, editing and the 3D pipeline. That needs to be combined with an interest in storytelling or at least an understanding of immersion rhythm, as your audience is in the experience, not just watching it.
LD: When you work with your designers, do you have guidelines for creating environments that might be viewed in many different ways: in-person, in-person/interactive potentially with wearables, or live-streamed?
JM: I have to consider many aspects to execute a well-delivered project. Firstly, we need to understand how long the visitor will be engaging with the content; this includes their flow around the space or venue and how interactive the experiences are. We are always hyper-aware of the visitor experience, which lets us create the smoothest and most engaging journey for them. If the experience is pulsed, free-flowing, a ride or a seated show, we always ensure that the technology is invisible and that the experience appears organic and self-sustaining.
LD: When you get a brief from a client, what are the main areas they need to be educated in? For example, are there areas of immersive entertainment that look expensive but are not? Are there new or old technologies that are affordable but overlooked? What does client education look like for you?
Tommy Lexen: Often we find ourselves educating clients before we even get a brief! To keep being at the cutting edge of the industry means we need to stay ahead of the game, look beyond the trends, innovate with the latest technologies, and hack existing systems to reimagine something new. So we try to plant the seed about what immersive art, storytelling and creative technologies can bring to an experience before the project actually commences.
We find its most useful to embed in the beginning stages while the client is conceptualizing a project with the wider stakeholders in order to get the most out of any collaboration.
Now, immersive entertainment is quite common in brand and marketing work, and there is little education needed in that sector. These days we’re working on how to educate whole new industries, like urban developers, yacht designers and climate advocates, on how immersive experiences can add value to their industries. It’s a pleasure to share our ideas and work with clients at the start of a project and help influence and extend their expectations for their projects.
JM: We can’t mention specific clients, but architects love working with us as we show them the opportunities offered by immersive design and spatial storytelling capabilities that they can build into their projects’ fabric.
We have such a huge experience of live shows that when we join a project, we help our clients see beyond the ‘show’, and we start the user experience at the start of the project - before the audience has even entered the space. We also look at how they can take elements of the show away with them, even if those elements are purely virtual.
LD: In your piece covering sustainability for immersive events you mentioned “re-skinning environments.” Do you mean an area of the metaverse or a physical set design? For designers used to creating one environment for a production and moving on, how are you changing this mind set and do you have examples of re-used environments?
TL: We define re-skinnable environments as a collection of multiple physical canvases whose digital content can be updated and changed without any physical alteration, making it a multi-use and future-proof installation. This isn’t just set design; it can include the whole spatial design of the space, the types of canvases and technologies being used, and the audience experience.
LD: What advice do you have for theatre design students who will graduate into a world where many productions will take place in the metaverse or a hybrid meta/in-person space? What technologies should they be learning?
TL: Even before the current hype of the metaverse, high-quality video and projection design have been a key element in theatrical designs. It’s now essential for theatre designers to understand how to collaborate with practitioners in these disciplines and integrate their work seamlessly into the design to create powerful storytelling in physical or hybrid spaces.
JM: Understanding how these mediums can enhance storytelling (in the same way lighting and sound can) is vital, but also having a good understanding of the capabilities of technologies like AR & VR, Flexible LED screens, and 3D projection mapping, as well as a basic understanding of the content creation process for animation and CGI (especially game engines like Unreal or Unity), would all help set them up to collaborate with immersive designers and prepare for a hybrid future.
LD: How do we keep the balance between the artist/s impact and the surrounding production? How do we enhance the emotion without bombarding the event with tech and stripping the human part of it?
JM: For us, this all comes back to storytelling and narrative. If you’ve created an impactful and engaging narrative and then allowed the narrative to dictate the use of the technologies that will enhance it the most, then you will not be “overwhelmed” by technology. In that way, immersive design is like all other art forms starting with the question of what story are you trying to tell? What do you want the visitor or audience to feel or experience? How is Immersion Rhythm being taken into consideration? We know we are one part of a greater whole and that we have opportunities to support our fellow artists in a larger collaboration, so it’s a pleasure to enhance, not distract.
LD: Have you ever had a client ask for something and you had to remind them that the audience came for the art not the tech?
TL: We have an iterative design process and what we call "immersive-design thinking". The key part is that Immersive Design-Thinking places the user/viewer/audience member at the center of the process to find truly human-centric solutions. We are constantly challenging our design with this thinking against the wider projects and clients' objectives and aims and iterating designs to develop the design in line with this. We often try things that don't work in the end, but testing and prototyping and iterating usually flush these issues out.
Thankfully, we have created these experiences for almost two decades, so our clients (usually) trust us. We also preview all our work within a bespoke VR environment so clients and stakeholders can come to their own conclusions about the feeling of the experience.
LD: Do you think immersive technologies will begin to level the playing field in terms of accessibility? If so, how?
JM: Absolutely. All our projects are designed with accessibility in mind, from World Expos and National Museums to live shows and theatrical entertainment. Technologies allowing closed captions and multiple languages are part of our narrative explorations. Defining sightlines and how the experiences are shaped from multiple height perspectives and consideration of audience flow are equally important in the physical space. Virtually it’s increasingly easy, although many headsets are still yet to be taken up on a large scale. In this sense, the pandemic has opened the world as much as it has closed it. We can see conferences coming online, museum experiences, and live shows, which all make these works accessible to wider audiences.
LD: Can you talk about some of your favorite projects? Why do they stand out?
JM: It’s so hard to pick favorites as Immersive’s work is so wide-ranging. We constantly work on different experiences and locations; the only thing that defines a constant is our creative process. In the past couple of years, it’s been exciting to go from designing eight highly educational installations at the Smithsonian National Air and Space Museum to creating more story-led installations for the Mobility Pavilion at EXPO 2020 and at the same time, creating a permanent hologram theatre performance in Xiangyang in China, as well as 15 hours of video art for a luxury yacht.
Last week, I took part in a design sprint with the Museum for the United Nations, and it was a great experience to work with behavioral scientists and climate advocates and scientists. Teamwork is the key to immersive experiences, and the power to bring significant change through inspiration is a crucial driver in everything we do.
TL: For me it must be the almost three years we worked on the Mobility Pavilion for EXPO 2020 in Dubai (that opened in October 2021). We told multiple stories across nine hero installations, including one deconstructed screen installation using 150 individual screens that together made up a 227 million pixel canvas, as well as telling a 3.5 minute story across a 23 meter (10:1), 14K resolution curved panorama screen. And, of course, commissioning it all remotely during a raging pandemic.
LD: What do you think will be the next big thing in immersive experiences?
JM: It’s hard to escape the rise of the metaverse! We will consistently see a more hybrid space where art is both a physical and virtual experience. In terms of technology, we think handsfree AR devices will create a seamless experience for audiences that will be adopted into the mainstream in a year or two. Imagine what Apple did for the mobile phone with the iPhone; that will happen again with the much-anticipated Apple Glasses.
TL: To add to that, I think Spatial NFT commissions will become more common as they give creators new opportunities to extend their lifespan and access to immersive experiences previously created for a particular event or space.
The simplest way to define what we mean by “Spatial NFTs” is pretty much NFTs with spatial information. It’s a blueprint for the creative and technical realization of an immersive experience in a physical space. It includes the art of content and the technical spec and manual needed to install it. And after their first installation, they can be dismantled, traded, sold, refitted, and reopened as initially intended, but anywhere.
JM: Content creation is getting easier and faster. Look at Unreal 5 and how it's empowering artists and designers or Dall-e, the artificial intelligence program that creates images from textual descriptions. All kinds of new creative outlets will allow teams to work together to make the world's next set of mind-blowing experiences. It's all about teamwork in this industry.
LD: What upcoming projects are you excited about?
TL: We’re currently in the final technical rehearsals for a permanent hologram theatre show in Xiangyang that will open next month after two years in the making. It’s an ambitious production combining live performance and creative technologies. It combines traditional theatre design, real-time animation, projection mapping and holographic content, all designed, directed, and delivered by our studio. It’s very exciting seeing it all come together and how well our collaboration between our global and local teams in China has worked.
JM: Looking ahead, Immersive is currently working on our own in-house project, redesigning the future of the physical museum experience with some fantastic stakeholders and partners. The project is still under wraps for now but will be a game-changer! Stay tuned.