“Here’s what often happens when people decide they’re going to use magic in a show,” says Teller, the one-named illusionist best known as the silent partner in the duo Penn and Teller. “They get three quarters of the way through the production—they’ve built the set, they’ve blocked everything, and now they go, ‘You know, it’d be pretty good to have magic in here.’ And then they call some hapless person in—and I’ve been this hapless person on occasion—and say things like, ‘We think it would be really nice for him to be standing centerstage and just turn into an elf.’”

News flash: Magicians are no longer content to conjure rabbits. They want to be seen as legit theatre artists in their own right. And increasingly they’re getting that chance, being called on to weave magic into the texture of theatrical stories rather than asked to tack it on as an afterthought. More and more directors and playwrights are collaborating with magicians from the get-go. Some notable upcoming projects include The Magic Play by Andrew Hinderaker (receiving its world premiere at Goodman Theatre in Chicago, Oct. 21-Nov. 20), and in New York producer Randy Weiner’s immersive theatre collaboration with Neil Patrick Harris, who in addition to being a triple-threat performer and seasoned awards-show host is also a skilled magician (the opening date will be in the fall; Harris is not slated to perform in the show).

In most of these new efforts, the idea is to use well-executed tricks as a way to advance the narrative and reveal something about characters, rather than simply pulling off a cool effect for its own sake. In a collaboration on par with a director’s back-and-forth with set or costume designers, or a playwright’s conversation with a dramaturg, the magician becomes a fully integrated part of the creative team. The result might look like neither a straightforward naturalist play nor a traditional magic show, but a kind of hybrid that synthesizes the emotional openness of theatre with the awe evoked by a good illusionist.

Among those leading the way are Teller, who has collaborated with playwright/director Aaron Posner on two Shakespeare productions: a “supernatural horror thriller” version of Macbeth (first produced in 2008 at Folger Theatre in Washington, D.C.) and a take on The Tempest that calls to mind Depression-era carnivals and traveling tent shows (it first appeared at American ­Repertory Theater in 2014 and was restaged at Chicago Shakespeare Theater last fall).

In those productions, the supernatural elements called for in the text—floating daggers, magically appearing bloodstains, vanishing sprites, conjurers with the power to make things levitate, and so on—were achieved via sophisticated illusion design not typically seen in your average Shakespeare production.

“When a character is supposed to be baffled by blood on their hands or a floating dagger, usually one either doesn’t see the dagger or you see it being held by another actor who isn’t supposed to be visible to us,” explains Posner. The effects created by Teller, on the other hand, have “the capacity to put the audience in the same mental place as the character—baffled, confused, surprised, horrified.” That justification in the text itself was “really the thing that was compelling to us in harnessing magic not as a specialty act but to be integral to the storytelling,” says Posner.

As codirectors, Posner and Teller made a strong effort to create magic that was awe-inspiring but never gratuitous. The floating dagger appeared via a mirror trick, underscoring Macbeth’s increasing susceptibility and false appearances. In The Tempest, the nimbleness of the spirit Ariel was established thanks to actor Nate Dendy’s facility with sleight-of-hand card tricks. “We take all of our cues from what Shakespeare wrote,” says Teller.

At the same time, both men believe magic can help make Shakespeare more accessible for audiences. “We’re both deeply populist,” says Posner. The same might be said for the Bard himself, who, for all his high-flown poetry, could never resist thrilling the groundlings. In Act Three of The Tempest, for instance, a stage direction calls for a banquet to vanish “with a quaint device.”

As Teller points out, “What he’s saying is, ‘Please do a magic trick here.’”

The broad appeal of magic is one reason why it’s frequently used as a storytelling tool by the House Theatre of Chicago, a storefront troupe specializing in original, pop-tinged work and bold stagecraft. The company’s first production, in 2001, was Death and Harry Houdini, a biographical portrait of the famed escape artist, written and directed by Nathan Allen. The magic tricks were devised by company member Dennis Watkins, who also played Houdini. In the years since that first production, the House has remounted and revised the show several times, most recently this past May and June (the production will travel to Miami’s Adrienne Arsht Center in 2017).

For Allen, the power of a well-executed magic trick lies in the way it can unite audience members by surprising them all at the same time—which has extra significance in a communal art form like theatre. “When you are looking at other audience members who are sharing your sense of wonder, and you can see the gaping mouth on the other side of the stage, you share that moment of, ‘Oh my God! What?!’ We want it to be a social experience.”

He believes that unified feeling is easier to achieve in a small space, even if pulling off Houdini’s death-defying stunts in close quarters becomes extra difficult. “Certainly you will never see the Water Torture Cell done in closer proximity,” he says. “You just won’t. It’s challenging, but we don’t want to be bigger than 200 seats. We want to give people a sense of the epic in an intimate space.”

In Death and Harry Houdini, that challenge has fallen on Watkins, who has appeared in and designed magic for every incarnation of the show. He describes the process as an exercise in reverse engineering. Allen will tell Watkins what he wants to see the character of Houdini accomplish—walk on broken glass, make people disappear, escape from a tank of water he’s been lowered into upside down while hanging from ankle restraints—and it will be Watkins’s job to make it happen, often only a few feet from the audience and in the round. This is “never the way a magician would design things,” concedes Watkins, citing the illusionist’s tendency to let the performing space dictate the scale of the magic: card tricks for small rooms, for instance, or daring physical stunts for Vegas-style amphitheatres where distance is a magician’s best friend. “In the last production of Death and Harry Houdini,” Watkins recalls, “Nate said, ‘We want to put a levitation in the show.’ And immediately I go, ‘Nate, it’s not possible. Look at this room.’”

But two months later, Allen was able to convince Watkins to give it a try, and surprisingly it worked. Though Watkins (ever the magician) won’t give away the secret of how he pulled it off, he does attest that the performance was “one of the most naked performance situations where I’ve ever seen a levitation work. It stretched me to think there’s a lot more possible with storytelling and magic than I would have imagined.”

Another thing that expanded possibilities, says Watkins, was the bump in production values as the company proved popular with audiences and budgets grew (though still nowhere near the size of what you’d find in a large nonprofit or commercial theatre). It wasn’t always thus: In Death and Harry Houdini’s first production in 2001, the House couldn’t even afford a real Water Torture Cell, so they made do with a cardboard cutout.

“I hung from those ankle things you put on when you put a bar across your door and do exercises,” Watkins says, “and we raised a blue curtain to look like water.” It wasn’t until the show’s 2003 revival that an actual tank was brought onstage so that contemporary viewers might feel the same hold-your-breath suspense Houdini’s audiences felt (and yes, it’s filled with actual water).

That doesn’t mean Death and Harry Houdini is a magic show per se. Story comes first to the creative team, which meant paring the magic down to key moments and cutting whatever felt superfluous, however impressive it might look. A bit with magically appearing doves, introduced in 2003, was discarded in 2013—not because working with live animals is a pain in the neck (“though it is,” Watkins admits), but because it didn’t advance the story.

A useful later addition, by contrast, was a scene in which Houdini, as a child, watches his dying father get sawed in half by the Grim Reaper. Far from seeming extraneous, says Watkins, “The way it feels for an audience member is you’re watching a kid experience death for the first time. And it becomes something that’s emotionally compelling as well as awe-inspiring.”

Using theatre and storytelling techniques to add emotional resonance to magic tricks is also a strategy employed by San Francisco–based performer Christian Cagigal. “While I love just doing a trick sometimes,” he says, “when I’m doing a show I’m always trying to create a narrative, a cohesive piece of theatre.”

His solo shows include Obscura—an exploration of death, war, and the devil—told with close-up sleight of hand. Another piece, Now and at the Hour, combines mind-reading tricks with the personal story of Cagigal’s father, a Vietnam vet with schizophrenia and PTSD. Beyond confounding the audience, the purpose of the show is to reflect on time, memory, and the mind.

Cagigal, who has theatre training and is a former member of the San Francisco Mime Troupe, likens his work to musical theatre, which also has two distinct yet interrelated entities: score and book. “Once upon a time in musicals, a loose plot strung together a bunch of pop songs,” he points out. “But then when you get to Sondheim—his mission is to push the story and character development through the song.” Cagigal aims for something similar in his work: magic that furthers a narrative, making both elements richer.

It’s not always an easy sell, especially with spectators inclined to approach magic as something to be dissected rather than enjoyed. “When I was doing a personal show like Now and at the Hour, I’m being vulnerable and open while sharing this piece of magic with someone onstage,” he says. “But sometimes, because it’s magic, they’re thinking to themselves, ‘This is a trick, so I won’t be fooled,’ and they won’t get with me on the same page. They’ll be antagonistic or funny during the routine. And I’m thinking, ‘Hey, I just opened up my soul to you! Don’t be a jerk!’”

But when it works, the moment of awe inspired by a successful illusion, like laughter for a good joke, can open minds that might otherwise stay closed. “Something that breaks up the trance in which we exist through our days can be a wonderful moment to digest a new idea. It’s the basis of all great art,” attests Cagigal.

Currently Cagigal is writing his own show about Houdini, though he plans to focus less on the magician’s famous stunts than on his early struggles to make a name for himself as a Jewish-­Hungarian immigrant in early-20th-century America. Cagigal sees contemporary resonances in Houdini’s defiance of instruments of authority, from straitjackets to supposedly impregnable jail cells, particularly in light of current protests against police misconduct. “Plus, every solo show is really about the person onstage,” admits Cagigal, who feels a personal connection to Houdini, as the son of immigrants (Cagigal’s father was born in Spain, his mother in El Salvador).

Andrew Hinderaker’s The Magic Play—which, after receiving its world premiere at the Goodman, will appear at the Olney Theatre in Maryland next spring—isn’t about real-life figures. Instead, the playwright uses a fictional gifted magician to explore aspects of his own life, as well as broader themes like loss and heartbreak. “I have an uncle who passed a few years back who was a fantastic set designer and amateur magician,” Hinderaker explains. “He was somebody who was consumed with his work…. And his life suffered a lot of consequences because of that.”

Something similar happens in the play, which starts out as an ordinary magic show built primarily around card tricks. As things progress, however, the magician’s messy past and shattered love life intrude. The tight precision required in sleight of hand is contrasted with flashier effects performed by the magician’s father—who uses a gimmicky deck of oversize cards—and another figure doing aerial stunts.

Actor/magician Brett Schneider, who plays the central role and helped develop the tricks in the piece, believes The Magic Play gets to the core of what attracts people to performing illusions in the first place. “As a magician, he’s very skilled and he knows how to control practically every situation he’s in,” says Schneider. “He knows how to handle any audience member, and he knows how to elicit responses. One of the fundamental questions of this play is, What does that do to a person who can simulate intimacy but is never truly vulnerable in front of someone else?”

Loss of control often pops up as a theme in Hinderaker’s plays, which include Suicide, Incorporated, about a company that lets clients outsource the writing of their suicide notes. In The Magic Play, Hinderaker ups the ante, stripping not just the characters of power but sacrificing some of his own control as well. Because the play is structured, at least at first, like a traditional magic show, it’s interactive. In one part of the show, the magician reads an audience member’s mind. In another instance, an audience member is brought up to the stage in order to help facilitate the spike in a bag trick (where the magician slams his hand down on several paper bags, in which there may or may not be an upright nail); it’s part audience participation, part trust exercise for the characters in the play.

“One of the things that’s really important to me is that the audience is invited to engage on a much deeper level than they normally would in a play,” says Hinderaker. To him, breaking the fourth wall heightens the play’s immediacy and introduces an electric atmosphere of unpredictability; the script is even designed at some points to accommodate different outcomes based on audience responses.

Another crucial part of the show’s development has been the playwright’s collaboration with Schneider, who’s been closely involved with the project since its inception. Schneider’s credentials include a membership at the Magic Castle in Los Angeles.

“Brett and I have been meeting on a weekly basis for about four years,” Hinderaker says. “The first year, Brett was constantly referring me to books, performances, magicians, and showing me a ton of pieces of magic that he admires.” Both men found that they were drawn less to showy stuff like assistants sawed in half than to elegant tricks such as the classic Sympathetic Cards, in which two shuffled decks somehow end up with their cards in identically matching order.

The magic in Hinderaker’s play is not just stand-alone special effects; it also comprises moments of dialogue and character development. So it was important for Hinderaker to write all the tricks. Schneider admitted that it was a challenge to both create and perform, but it’s also been a creative booster.

“A lot of the times when you’re thinking like a magician, you’re immediately giving yourself limitations because you know what’s been done before and what the technical parameters are,” he says. “But when you’re thinking like a playwright, especially one as gifted as Andrew, none of those limitations apply.”

The Magic Play has expanded Schneider’s idea of what’s possible in both his acting and his magic. Not to mention, he adds, “We’ve been able to successfully portray every moment that [Andrew’s] written so far,” he says.

This foray into magic is part of Hinderaker’s fascination with capturing in the theatre that visceral thrill spectators get at other types of live events. A previous play of his, Colossal, tells the story of a catastrophic football injury using dance, a drum line, and full-contact hits. The tricks and interactivity in The Magic Play—and really, any magic-theatre work—are another way of exploring what “live” theatre truly means.

“If we’re going to talk about something that’s live, and especially if we’re going to use that bullshit term about a conversation with the audience—the most one-sided conversation that exists on the planet—if we’re ever going to embrace that, then it can’t be pure control and manipulation,” Hinderaker attests. “If we’re not willing to loosen that grip, I question whether we’re ever able to create something that feels like magic.”

Zac Thompson is an arts journalist and critic based in New York City (formerly in Chicago). His byline has appeared in the Chicago Reader and Chicago magazine.