Is This Film the First True Live-Action Virtual Reality?

THE AGE OF the “live-action virtual reality experience” hasn’t been long, but it’s been busy. From the New York Times to the Sundance Film Festival, from the UN to the NBA, there’s no shortage of offerings—and that’s not even counting Facebook and YouTube, both of which are investing in VR video in a big way.

There’s just one problem: what’s being called “live-action VR” isn’t exactly that. It’s 360-degree video, and as such has one very key limitation: you can’t move inside it. You can swivel your head, and you can look up and down, but you can’t lean forward to change your perspective, or peek around an object. You have three “degrees of freedom” (what’s known as 3DOF), as opposed to the six that you have in computer-generated experiences like a videogame or an animated film. And despite a few companies that aredigitizing people and placing them in 3-D environments, we have yet to see true live-action VR that places you in navigable space.

Until now.

Photography company Lytro, which early last year shifted from still cameras into the world of VR, has unveiled what it’s calling the first 6DOF 360-degree film. Moon lasts less than 45 seconds, but it shows a peek at what simply hasn’t been possible yet. It is, VP of engineering Tim Milliron says, Lytro’s “coming-out party.” “We’ve been working on the tech a long time,” he says, “and this is proof that you can do it.”

In the short, an astronaut standing on the surface of the moon delivers Neil Armstrong’s famous 1969 speech. “That’s one small step for man,” he says, the Earth hovering in the sky over his shoulder, “one giant step for mankind.” Oops.

“Cut!” a voice over your shoulder yells. The lights go up. You’re not on the moon at all—you’re on a soundstage, and when you turn around, Stanley Kubrick is sitting in a director’s chair looking pissed. “I’m an actor, not an astronaut,” the guy in the space suit says. It’s quick, but it’s cute. (And if you’re still hungry for the-moon-landing-is-a-Kubrick-filmed-hoax conspiracy theories, we can happily recommend Room 238 and Operation Avalanche.)

More Reality with Less Work

But what matters is that through it all, it’s simply more realistic than a mere 360-video 3-D video. There’s no stitching. If you lean forward, your perspective on the astronaut and the lunar module changes. The lighting is view-dependent: when your perspective changes, so do surface reflections. It’s a quantum leap forward for the medium, as well for the words we use to describe it. (It’s also already dated; I also saw more recent test footage from Lytro that I can’t discuss, but is hands down the most lifelike thing I’ve ever seen in VR.)

Lytro thinks of virtual reality as a balance between realism and immersion, with true immersion only being possible with six degrees of freedom. Videogame engines, Milliron says, are high-immersion, but low-reality; 360-degree videos are the opposite, shackling photorealism with a stationary forced perspective. However, Lytro’s Immerge camera system, which the company used to make Moon, seeks to produce work that’s both high-realism and high-immersion—what “VR nerds think of as table stakes,” Milliron says.

Lightfield camera arrays like the Immerge—which captures not just a flattened view of a scene, but all the light rays within a scene, allowing for full three-dimensional reconstruction of an environment—are still rare. Other companies have demonstrated the ability to capture a lightfield, but none have harnessed it to create narrative content.

Maybe most astonishing, the workflow is comparable to conventional 360-degree videos. After a two-day shoot, the post-production process amounted to a single person working for eight weeks. (The lunar lander and moon surface were computer-generated, and produced by an outside firm.) And that was with an outdated version of Lytro’s camera and processing tools; Milliron estimates that in the future a crew of two or three could make a much longer movie in the same amount of time.

But whose crew will that be? Not Lytro’s. “We don’t want to be content producers,” Milliron says. “This is just to put the system through its paces.” The company is finalizing discussions with some VR production houses, and will manufacturer Immerge arrays to suit the needs of whoever will be using them.

At least until the next quantum leap happens. “I don’t think anybody knows how VR is going to be shot in a year or two,” Milliron says.