For Avatar: The Way of Water, a new cinematic journey from 20th Century Studios and Lightstorm Entertainment, Academy Award-winning filmmaker James Cameron and his team developed new filmmaking technology to capture the actors’ performances underwater—something that had never been done before in the history of cinema. “The key to it was to actually shoot underwater and at the surface of the water so people were swimming properly, getting out of the water properly, diving in properly,” Cameron says. “It looks real because the motion was real. And the emotion was real.”
First, the crew constructed a tank at the Manhattan Beach Studios in California, where Cameron and producer Jon Landau’s production company is located. The tank’s enormous size allowed Cameron to replicate real-world oceanic conditions. At 120 feet long, 60 feet wide, and 30 feet deep, it could hold more than 250,000 gallons of water. “That became our complete Swiss army system,” Cameron says. “We could do waves breaking on the shore and have people trying to get out of the water while they’re getting hit by waves. We could create wave interaction with the creatures and people surfacing, getting hit by a wave, trying to say their lines, and trying to breathe at the same time.” A propeller system dubbed “the racetrack,” which included two six-foot- diameter ship propellers, was used to drive the current in the tank. “It was only a 10-knot current,” he explains, “but we were able to make it look much faster for the film.”
For the performance capture technology to work underwater, the water needed to be crystal clear—meaning scuba gear wasn’t a viable filming option. “You can’t have a lot of air bubbles,” Cameron explains. “Every one of those air bubbles is a little wiggling mirror, and the system that’s trying to read all the marker dots on the actor’s body so it can capture their motion can’t tell the difference between a marker dot and a bubble.” That left Cameron and his crew with one option: “Everybody who was working in the tank was holding their breath,” the filmmaker says. “If there was somebody down there holding the light, they were holding their breath. If they were operating a camera, they were holding their breath. The actors, of course, had to be holding their breath.”
Landau explains, “We didn’t want to simulate it. We didn’t want to do what people called ‘dry for wet.’ We wanted to put Zoe Saldaña, Sam Worthington, Sigourney Weaver, and others in water—to have them feel the current, the movement, the waves, and to give a performance. No one has ever done performance capture underwater.”
To give compelling performances underwater, the cast studied free diving with internationally recognized expert Kirk Krack. “The actors really enjoyed it,” Cameron says. Indeed, Worthington says, “Jim is always pushing technology and how to uniquely tell a story in a way that has never been tried before. I enjoy being at the forefront of that with him, being tested and challenged, physically and emotionally.”
With the tank built and the cast and crew trained in free diving, Cameron was able to bring his grand vision to life. “What we essentially wound up with was a volume [aka performance-capture stage] for underwater and a separate volume for the air,” he says. “Those two volumes had to sit right on top of one another with only an inch in between. The computer’s taking data from one volume, data from the other volume, and in real time integrates all that information and shows me on my virtual camera people coming and going, swimming up, getting out onto a dock, or diving in and swimming underwater. It was two completely separate methods of capture being fused together. Obviously, the software to do that took quite a while to get worked out, but the end result was amazing.”
Performance capture for Avatar: The Way of Water began in September 2017 and ran for roughly 18 months, with Cameron and the cast working on scenes for all four of the sequels. In the edit bay, after Cameron and the editorial team picked the best performances for each moment of a given scene, Cameron then employed a revolutionary virtual camera to create the specific shots. The virtual camera allowed him to shoot scenes within his computer-generated world, just as if he were filming at a real location or on a Hollywood soundstage. Through this virtual camera, he would see not Saldaña, Worthington, or Weaver, but their giant blue characters in the world of Pandora. “I could see everybody where they’re supposed to be, above or below the water, and I could talk to them over the diver address system,” he says. “They were acting to real-time direction based on what I was seeing on the virtual camera.”
Once the virtual camera shots were edited into cut sequences, the shots and performances were delivered to the visual effects experts at visual effects company W?t? FX in New Zealand. Under the direction of senior visual effects supervisor Joe Letteri, in collaboration with Lightstorm’s Richard Baneham, the W?t? FX artisans worked to preserve the nuances of each actor’s performance.
“We look at every actor and every performance at a frame-by-frame level to make sure it matches,” explains Letteri. “To me, it always comes down to the characters and the ability to be with them, to be in the moment with them, to see their performances, to understand what they’re feeling and what they’re going through. That emotional connection is always what you’re going for.” Ultimately, Landau says, “W?t? FX is realizing our shots at a much higher-fidelity performance and photo-real quality. When we made the first film, I told people we needed to be photographic; we didn’t need to be photo-real, because no one could tell me what’s ‘real’ on Pandora. But now, because we have so many scenes that integrate both live-action and [computer-generated] characters and elements, we had to raise the bar to deliver something that is 100 percent photo-real.”
Undoubtedly, the underwater performance capture technologies and techniques that Cameron and his creative team crafted for Avatar: The Way of Water will ripple across future Hollywood productions. “It’s an incredibly complex problem, but it’s not like we were starting from scratch,” Cameron says. “We were doing water simulations back on Titanic, but [now] it’s taking it not just to the next level, but up five levels. The beauty of it is, if you can solve water for this movie, you can do all water anytime—until the end of time. These tools become incredibly important for the effects industry at large.”