Most of us walk around with a computer in our pocket—one that’s able to take and filter photos, capture and edit video, find us a date on Tinder, and look up the nearest sushi place (and call people too, if you’re feeling retro). It’s a kind of magic at our fingertips that we now take for granted. But if you regularly use technology to make theatre, you may have more appreciation for new breakthroughs that allow you to incorporate tech into your stage productions more easily and cheaply than ever.
“When we first started it was, like, so unbelievably analog; we had to haul these TVs around, literally!” explains Marianne Weems, artistic director of New York City’s 22-year-old Builders Association, recalling how difficult it was to use multimedia onstage in the ’90s. “Now we can run everything off basically one computer… The fact that it’s become so available and accessible, and everyone can make a movie with their phone now, has made it even more fluent than when we began.”
As troupes like Builders or the Wooster Group demonstrate, incorporating technology into stage work isn’t new. And video design is now as ubiquitous onstage as sets and lights. But when The Curious Incident of the Dog in the Night-Time made waves on Broadway last season, it wasn’t just the story of a boy on the autism spectrum that sent critics gasping; it was also the immersive use of projections on four walls, which interacted with the actor.
For Kevin Cunningham, artistic director of 3LD Art & Technology Center in Lower Manhattan—which produces and presents technologically driven work, as well as provides training in multimedia—Curious Incident is a mainstream manifestation of work that was pioneered by independent designers and collectives. And most don’t need a Broadway budget. “You can build a really large-scale complex show on your laptop with software that costs $350,” he says.
Most of this work doesn’t ignore performers or audiences; instead it connects those bodies using multimedia. Call it interactive, responsive, or immersive technology; whatever it is, it’s cool.
Cool but not chilly, Cunningham emphasizes. “Technology in the performing arts has to be responsive, because in the live performance, everything has to work, and the human body has to be able to interact seamlessly and smoothly with technology. So the first thing that has to go by the wayside is the idea that technology is unnatural and cold. It’s just another form of human expression.”
The six projects featured here represent just a small cross-section of the brave new world of interactive technology onstage. From creating virtual worlds on smartphones to live moviemaking, from wearable tech to immersive projections, high-tech and low-tech, these productions blur the line between theatrical disciplines. Is it set design? Lighting? Video? Dramaturgy? All of the above?
Elements of Oz, the Builders Association
Imagine going into a theatre where the usual pre-show talk isn’t “Turn your phones off” but rather “Leave your phones on.” That’s what audience members experienced at Elements of Oz, a new work that premiered last year at New Jersey’s Montclair State University and will return to 3LD in the fall. Theatregoers were next invited to download an app onto their smartphones and tablets, also called “Elements of Oz.” And that’s where things got interesting.
The show follows the backstage story behind the Wizard of Oz film, with the performers using live cameras, actors, and some iconic costumes to perform scenes from the film in the order that they were filmed in real life, using the real shooting script from the movie. According to Weems, who directed the show, Elements of Oz isn’t an adaptation of the iconic Baum tale but rather a story about the “physical labor of making a movie.”
But what about the special effects? How do you create a tornado onstage? That’s where the smartphones came in. Elements of Oz used a technique called augmented reality where the real world as viewed through a mobile device is then supplemented by digital imagery (think Snapchat lenses). Augmented-reality designer John Cleater and his team were able to design an AR app for the show using a game development software called Unity. The result created such effects as the tornado, poppy fields, and flying monkeys.
So during the tornado scene, for example, audience members could hold up their phones and see a tornado superimposed over the action onstage. When that tornado flew into the house, they could move their phones and follow it; every member of the audience had a different vantage point for the effects, depending on where they were sitting. Other AR effects from the app included Munchkin laughs that rippled throughout the audience. “It was kind of creepy and cool and everybody loved that,” says Cleater proudly. “In the end they wanted more [AR]!”
Apart from giving tech addicts an excuse to keep their phones on, AR enhanced the show.
“Everybody spends half of their time looking at their devices,” says Cleater, who admits that he doesn’t own a smartphone. “It just seems Shakespearean to put that onstage. It’s not the real world. It’s not like we’re doing some kind of abstract new media thing. We’re just putting reality onstage, in a way.”
ADA/AVA, Manual Cinema
Technology doesn’t necessarily mean a MacBook. Though Manual Cinema is only six years old, this Chicago company has already become well known for their unique blend of shadow puppetry, live action performance, and cinematic techniques, accomplished with overhead projectors. ADA/AVA, originated in 2013 and running at the Edinburgh Festival Fringe Aug. 3-29, is about a pair of close-knit twin sisters; when one dies, the other must learn how to live alone. To tell that story, the company uses three overhead projectors, more than 300 paper and acetate puppets, two live actors playing the sisters, and a live band scoring the story. There is also a digital camera backstage, which shoots both actors and puppets and melds them together onto a large overhead screen.
In short, every night, Manual Cinema makes both a movie and a play at the same time, using paper and projectors. The final product is a wistful, wordless mix of Alfred Hitchcock and New England Gothic.
“[The overhead projectors] became the defining factor of our aesthetic, and working in silhouette,” says co-artistic director Julia Miller, who played Ada in ADA/AVA while also puppeteering. “We really started with one overhead projector [in 2010], and each show we added another and another and another. Each time we added another overhead projector, it was essentially like adding another camera, because it doubled the number of shots.” She means shots in the cinematic sense: tight, medium, wide. To change scenes, puppeteers switch between projectors during the show.
Manual Cinema does all this in full view of the audience, who are able to see the band playing onstage, the puppeteers moving the puppets, and the actors putting their costumes on. Afterward, audience members can come onstage for a closer look. “It’s not a passive experience,” says Miller. “We’re kind of showing you all this stuff and asking you to watch this film but also wonder, ‘Oh, how are they doing that?’ Giving you agency as an audience member also became really exciting and important to us.”
Because the movie is being made in real time, it’s also easy to hide mistakes if the actors ever fudge a cue or a puppeteer doesn’t switch a scene fast enough. “We make 15 mistakes per show,” admits co-artistic director Kyle Vegter. “Performing one of these shows feels like you’re part of a living organism.”
A Very Electric Christmas, Lightwire Theater
New Orleans’s Lightwire Theater builds their family-friendly shows on a very simple foundation: electroluminescent, or EL, wires, which are attached to sculptures lined with black fabric. For their holiday show A Very Electric Christmas, about a young bird who gets separated from his family, Lightwire incorporates such sculptures as a nutcracker, a rat king, birds with flapping wings, and a Christmas tree. During performances, the house and stage lights are turned off completely, and the only things the audience sees are dancing sculptures made of light. Lightwire shows have no dialogue. More startingly, there is no light board; everything is controlled by the performers onstage.
“You are your light,” says Ian Carney, artistic director of Lightwire. Each sculpture comes equipped with a mini light board on the chest, so that performers can turn themselves on and off, or dim the light as needed. This means the dancers have flexibility in how they use the technology in their performance. “For us, being able to dim the EL wire down, brighten it up—these (along with the dancer’s movements) give you this organic sort of thing. And so [the dancers] use their fingers on their chest like they’re playing a small piano. If they want to cower and be scared, they can dim themselves down… They can lunge forward and brighten up, add colors. They have multiple switches.”
All of the sculptures at Lightwire are built in-house with the help of the show’s cast. And it’s not a high-tech affair: Materials include aluminum rods, epoxy clay, PVC pipes, and duct tape. The lights are powered by AA batteries. But these humble materials don’t stop the Lightwire team from dreaming big. Previous shows have featured dinosaurs and 16-foot-tall birds. The theatre has been featured on “America’s Got Talent” and has toured worldwide. And while all the flashing lights may attract attention, Carney is adamant that every show also has a solid dramatic foundation.
“As much as we thought the medium was cool, the absolute most important thing is story,” he maintains. “Without good storytelling, it’s an hour and a half of tediousness. We like to say that coolness is going to get you five minutes, and after that you better be interested in a character. If we can make them forget about the coolness of the EL wire as quickly as possible, that’s the main thing.”
A Very Electric Christmas will tour the country Nov. 4-Dec. 30. Lightwire’s current show, Moon Mouse, is touring in China through July 10.
Half Life, Cloud Eye Control
The Los Angeles-based collective Cloud Eye Control has interactive media and performance as part of their mission. Founded by three graduates of CalArts’s Integrated Media program (Chi-Wang Yang, Anna “Oxygen” Huff, and Miwa Matreyek*), the company’s shows consider how human beings function in an increasingly tech-heavy world. Their newest show, Half Life, was inspired by the 2011 Fukushima Daiichi nuclear disaster, and had its world premiere at REDCAT in Los Angeles in 2015; the company is in talks for an engagement at San Francisco Museum of Modern Art.
In exploring how two women respond to an unnamed ecological disaster, the show primarily uses projections, with four movable scrims and two rear-projection screens. At various points in the show, the actors are completely surrounded by projections, peeking through the window of an apartment building, walking in a forest, or lost in the middle of an ocean. “So much of our lives are about screens and information that’s ephemeral,” says Matreyek. “So what does it mean for that to be an environment that you’re inside of, with projections?”
But because the actors were reacting to the images via preset cues from the live performance software QLab, the group wanted to add components to Half Life that could be controlled by the onstage performers. “Everything is this kind of pretend interaction,” says Matreyek. “It’s cue-to-cue, but it’s not being reactive, so we wanted to put in a few things that were actually reactive.”
Enter face tracking: Using a live camera onstage and facial mapping, audiences could see, for instance, boils erupt onto actress Anna Oxygen’s face. Cloud Eye Control ran preliminary tests using projections, then brought in video designer Jackson Campbell to build the face-tracking functionality. For Matreyek, it added an extra layer of interactivity that supported the dramaturgy of Half Life.
“It’s a nightmare moment of a hypochondriac,” she explains of the boil-erupting scene. “The audience doesn’t know if it’s her imagination that’s seeing it or she’s actually seeing it. She thinks she’s getting sicker and sicker.”
YOUARENOWHERE, Andrew Schneider
A one-person show can be a self-indulgent experience, as if it’s designed more for the performer than for the audience. So when video-designer-turned-performance-artist Andrew Schneider was creating YOUARENOWHERE (which can be read either as “you are now here” or “you are nowhere”), he wanted to stand out. The result—a meditation on existential angst, relative time, and multiple universes—won an Obie after running last year at Performance Space 122, and is now on a worldwide tour. And technology is the foundation. Schneider built a majority of YOUARENOWHERE using such software as Ableton Live and Isadora, which ensured that sound, lights, video, and text could be controlled simultaneously. He didn’t so much write a play as create the text in conjunction with visuals.
“I don’t even like talking about technology as a separate thing,” Schneider says. “It’s the same as everything else. It’s the same as the text. We’re using all these things to tell the story at the same time.”
Not only are audiences listening to Schneider speak his monologue, they’re also witnessing quick flashes of light, feeling sound vibrations under their feet, hearing Schneider’s voice become more distorted as he speaks, and, in one remarkable moment, believing they’re in an alternate universe. Most of the light and sound cues are set on a computer timeline to maintain the precision; Schneider wears an earpiece that tells him when the next big cue is coming.
He can control some of it too by using the microphone attached to his face. “In different times during the show, the volume of my voice will affect the brightness of the light,” he says. At one point, the words he speaks are automatically recorded live, then played back.
It all sounds very complex and random. So what does it all mean?
“I’m interested in what technology can do to further narrative or sharing an experience in the room with people that we haven’t been able to do,” Schneider says. He explains that YOUARENOWHERE is intended to play with the audience’s perception of time and to overwhelm their senses. “The first half of the show is meant to be this sensory overload experience, and I can’t do that with traditional infrastructure.”
And it seems like audiences are responding well to technology-driven, sensory-overload experiences; both YOUARENOWHERE and Manual Cinema’s ADA/AVA were nominated for a 2016 Drama Desk Award for unique theatrical experience (the winner was announced after press time).
The Nether, Woolly Mammoth Theatre Company
The Nether is a story about characters who enter a virtual world inhabited by avatars of little girls and the pedophiles who love them. In Woolly Mammoth’s production of Jennifer Haley’s unsettling and morally ambiguous play (which ran April 4-May 1), projection designer and Woolly company member Jared Mezzocchi was tasked with creating a stage set entirely out of projections. That meant working with set designer Sibyl Wickersheimer to map out the looks of the different rooms in the play (including picking out wallpaper and crown molding). It also meant creating a world that wasn’t static, that could respond to the actors onstage in real time.
“This is something we discovered throughout tech: that a literal depiction of reality would not be something that a creator of a virtual universe would want,” says Mezzocchi. “There’s a lot of expressionistic use of projection, where the world will basically respond and travel with the psychology of the character. So it’s not so much a ‘hold for tech, and talk through it with the actors,’ it was more, ‘I think the world needs to follow the energy that the actor was producing.’” In other words, “Media is the puppet and the actor is the puppeteer.”
The projection effects included pixelating an actor’s body as they’re standing behind a projection screen; shadows following the actors across the stage; and having the virtual world of the Nether fall onto the actor at climatic moments. Seven projectors were used and all of the effects were cued; there were no sensors or motion detectors. But if nothing else, Mezzocchi showed that an ambitious and immersive-feeling projection design can be done inexpensively; Mezzocchi used the popular program Isadora, which costs $450.
“We’re moving in a direction where the infrastructure is more accessible to more people in more theatres,” says Mezzocchi. He says he expects the trend to continue, as theatres realize that technology is more than a flashy gimmick; it can help tell the story, particularly if technology is the story’s theme. “We actually can help with the storytelling. It’s not just a special effect when you need a strobe light or haze. If theatre is holding up a mirror to society, I think it’s really important that sound and video become key components to storytelling.”
*A previous version of the article did not list the three cofounders of Cloud Eye Control. At the request of the artists, we’ve added their names here.
