Admitting Meta CEO Mark Zuckerberg, Starburst in its current form is “extremely impractical”. But for a company that wants to give its users virtual experiences that are almost indistinguishable from the real thing, these huge VR binoculars are nevertheless an important development.
To really blur the line between the physical and the virtual — or pass the “visual Turing test,” as some researchers say — Meta has to overcome some serious hurdles. Future headsets should be slimmer than the headsets we have now, yet more capable. And the screens in it need to be sharper, smarter and brighter than anything out there now.
That’s why Starburst is built around a big lamp – it’s one prototype, meant to tackle one big problem. And it’s not alone.
“The goal of all this work is to help us identify which technical paths allow us to make meaningful enough improvements that we can approach visual realism,” Zuckerberg told reporters during a presentation.
That veracity is a critical part of his vision of the metaverse: an immersive “embodied internet” where users feel like they are inhabiting a space rather than just looking at it. But despite the wave of metaverse hype Zuckerberg launched after laying that vision last yearMeta’s prototypes provide a tangible sense of how far the company is from delivering on that promise.
First, the company needs to figure out how to make everything we see through a headset more detailed.
Think of your TV or your computer monitor: the higher the resolution, the sharper and more realistic the things displayed on it look. But the small screens in today’s VR headsets can’t come close to that sharpness – they’re too few pixels, stretched over too large a space.
Another prototype, Butterscotch, more or less solves the problem. It’s bigger than anyone would want to wear for very long, and “nowhere near shippable” according to Michael Abrash, chief scientist at Meta’s Reality Labs division. Still, the images it produces are detailed enough that a wearer can read the bottom 20/20 line of a virtual vision map – not bad, compared to the blurry spots seen. through a Meta Quest 2†
The catch? Researchers had to narrow the field of view to about half of what you would see through the Quest 2. That is, if you look through Butterscotch, you see less of the virtual world in front of you – but what you can see looks very clear. Not a great trade-off, but Abrash admits it will take at least a few years for the right kinds of screens to exist.
“There are currently no display panels that support anything close to the full-field retinal resolution of VR headsets,” he said.
Another prototype, called Half Dome, was first conceived in 2017 and is now in its third overhaul. In this headset and others like it, Reality Labs researchers have refined so-called “varifocal” lenses — lenses that move physically and automatically to help the wearer’s eyes focus on virtual “objects” in front of them.
If you were wearing a traditional VR headset, you would find that the focal length is a few feet in front of you. Try bringing an object, such as a virtual handwritten letter, closer to your face and you will find that you cannot read it.
In such a situation, your real eyes focus just fine – the problem is that your view of the world is naturally a bit farsighted. Varifocal lenses are thus like glasses with a life of their own, moving to keep virtual objects sharp, wherever they are.
Meta has been experimenting with these lenses for nearly five years, the company says, and despite once claiming that they almost “ready for prime time”, they haven’t appeared in any headset you can buy right now. And for now, that doesn’t seem to be changing.
“Even if you sometimes have a prototype that seems to work, it can take a while to put that into a product,” Zuckerberg says. “We are working on it.”
One final prototype Meta showed reporters — dubbed Holocake 2 — bringing home Zuckerberg’s point.
Unlike other experimental headsets Meta showed, Holocake 2 is completely portable and functional – it can be plugged into a computer and run existing VR software without any problem. And because of the specific way researchers designed the optics, Holocake is the thinnest, lightest VR headset the company says it has ever made.
But even that doesn’t mean Holocake will be ready to hit store shelves anytime soon. Unlike more conventional VR headsets, Holocake 2 uses lasers as light sources instead of light-emitting diodes or LEDs. (You know, the stuff in some of your light bulbs).
“As of today, the jury is still out on finding a suitable laser source, but if that proves feasible, there will be a clear path to sunglasses-like VR displays,” said Abrash.
The fact that these prototypes exist is proof that these problems can be tackled individually – if not always elegantly. The real problem, though, is building a single headset that addresses all of these areas while being comfortable and energy efficient. And researchers suspect the end result could resemble a concept design called Mirror Lake.
While it doesn’t exist as a working prototype (and probably won’t for a while), Mirror Lake packs much of that visual advancement — plus a display that lets bystanders see the wearer’s eyes and face — in a headset that looks like a pair of ski goggles.
Douglas Lanman, director of display systems research at Meta’s Reality Labs division, also called Mirror Lake the company’s first “mixed reality” concept, referring to a type of wearable display intended to combine digital objects and environments with your view. on the physical world.
It would be a “game changer for the visual VR experience,” Abrash says. Now Meta just has to make it – or something like that.
Meanwhile, the company faces other headwinds.
Meta’s revenue growth begins to slow and Reuters reported last month that the Reality Labs division could not afford to continue certain projects. Hiring at the company has also been delayed, although spokesperson Elana Widmann said Meta “has no plans for layoffs at this time.” And while the company was expected to release augmented reality glasses codenamed Project Nazare in 2024, those plans were said to have been demolished to make it a demo device.
“We are evaluating and putting energy into key priorities across the company, especially as they relate to our core businesses and Reality Labs,” Widmann said in an email.