People are already confused about virtual reality (VR) , augmented reality (AR) , mixed reality (MR) , 360-degree video and heads-up displays. It will get worse before it gets better.
Words matter. And as a stickler for accuracy in language that describes technology, it pains me to write this column.
I hesitate to expose the truth, because the public is already confused about virtual reality (VR) , augmented reality (AR) , mixed reality (MR) , 360-degree video and heads-up displays. But facts are facts. And the fact is that the technology itself undermines clarity in language to describe it.
Before we get to my grand thesis, let’s kill a few myths.
Silicon Valley just produced a mind-blowing new virtual reality product. It’s a sci-fi backpack that houses a fast computer to power a high-resolution VR headset. Welcome to the future of VR gaming, right?
Wrong.
While the slightly-heavier-than-10-pound backpack is conceptually similar to existing gaming rigs, it’s actually designed for enterprises, as well as healthcare applications. It’s called the Z VR Backpack from HP. It works either with HP’s new Windows Mixed Reality Headset or with HTC’s Vive business edition headset, and houses a Windows 10 Pro PC, complete with an Intel Core i7 processor, 32GB of RAM and, crucially, an Nvidia Quadro PS2000 graphics card. It also has hot-swappable batteries.
Will HP’s new enterprise-ready VR backpack deliver mixed reality, augmented reality or virtual reality? The answer is yes!
To me, the biggest news is that HP plans to open 13 customer experience centers around the world to showcase enterprise and business VR applications. If that surprises you, it’s because the narrative around VR is that it’s all about immersive gaming and other « fun » applications. It’s far more likely that professional uses for VR will dwarf the market for consumer uses.
All of these technologies have been around for decades, at least conceptually. Just now, on the brink of mainstream use for both consumer and business applications, it’s important to recognize that different people mean different things when they use the labels to describe these new technologies.
A Singapore-based company called Yi Technology this week introduced an apparently innovative mobile gadget called the Yi 360 VR Camera. The camera takes 5.7k video at 30 frames per second, and is capable of 2.5k live streaming.
Impressive! But is 360-degree video « virtual reality »? Some (like Yi) say yes. Others say no. (The correct answer is « yes » — more on that later.)
Mixed reality and augmented reality are also contested labels. Everyone agrees that both mixed reality and augmented reality describe the addition of computer-generated objects to a view of the real world.
One opinion about the difference is that mixed reality virtual objects are « anchored » in reality — they’re placed specifically, and can interact with the real environment. For example, mixed reality objects can stand on or even hide behind a real table.
By contrast, augmented reality objects are not « anchored, » but simply float in space, anchored not to physical spaces but instead to the user’s field of view. That means Hololens is mixed reality, but Google Glass is augmented reality.
People disagree.
An alternative definition says that mixed reality is a kind of umbrella term for virtual objects placed into a view of the real world, while augmented reality content specifically enhances the understanding of, or « augments, » reality. For example, if buildings are labeled or people’s faces are recognized and information about them appears when they’re in view, that’s augmented reality in this definition.
Under this differentiation, Google Glass is neither mixed nor augmented reality, but simply a heads-up display — information in the user’s field of view that neither interacts with nor refers to real-world objects.
Complicating matters is that the « mixed reality » label is falling out of favor in some circles, with « augmented reality » serving as the umbrella term for all technologies that combine the real with the virtual.
If the use of « augmented reality » bothers you, just wait. That, too, may soon become unfashionable.
And now we get to the confusing bit. Despite clear differences between some familiar applications of, say, mixed reality and virtual reality, other applications blur the boundaries.
Consider new examples on YouTube.
One video shows an app built with Apple’s ARKit, where the user is looking at a real scene, with one computer-generated addition: A computer-generated doorway in the middle of the lane creates the illusion of a garden world that isn’t really there. The scene is almost entirely real, with one door-size virtual object. But when the user walks through the door, they are immersed in the garden world, and can even look back to see the doorway to the real world. On one side of the door, it’s mixed reality. On other side, virtual reality. This simple app is MR and VR at the same time.
A second example is even more subtle. I’m old enough to remember a pop song from the 1980s called Take On Me by a band called A-ha. In the video, a girl in a diner gets pulled into a black-and-white comic book. While inside, she encounters a kind of window with « real life » on one side and « comic book world » on the other.
Someone explicitly created an app that immerses the user in a scenario identical to the « A-ha » video, wherein a tiny window gives a view into a charcoal-sketch comic world — clearly « mixed reality » — but then the user can step into that world, entering a fully virtual environment, with the exception of a tiny window into the real world.
This scenario is more semantically complicated than the previous one because all the « virtual reality » elements are in fact computer-modified representations of real-world video. It’s impossible to accurately describe this app using either « mixed reality » or « virtual reality. »
When you look around and see a live, clear view of the room you’re in, that’s 360-degree video, not virtual reality. But what if you see live 360 video of a room you’re not in — one on the other side of the world? What if that 360 video is not live, but essentially recorded or mapped as a virtual space? What if your experience of it is like you’re tiny, like a mouse in a giant house, or like a giant in a tiny house? What if the lights are manipulated, or multiple rooms from different houses stitched together to create the illusion of the same house? It’s impossible to differentiate at some point between 360 video and virtual reality.
Purists might say live, 360 video of, say, an office, is not VR. But what if you change the color of the furniture in software? What if the furniture is changed in software to animals? What if the walls are still there, but suddenly made out of bamboo? Where does the « real » end and the « virtual » begin?
Ultimately, the camera that shows you the « reality » to be augmented is merely a sensor. It can show you what you would see, along with virtual objects in the room, and everybody would be comfortable calling that mixed reality. But what if the app takes the motion and distance data and represents what it sees in a changed form. Instead of your own hands, for example, it could show robot hands in their place, synchronized to your actual movement. Is that MR or VR?
The next version of Apple maps will become a kind of VR experience. You’ll be able to insert an iPhone into VR goggles and enter 3D maps mode.
