Made with Apple’s ARKit, Amazon’s new augmented reality (AR) mobile app shows you can do AR sizing without needing depth-sensing camera. Sorry, iPhone X.
When Amazon announced its new augmented reality (AR) mobile app for iOS last week, it stressed visual issues and said it allowed shoppers « to make sure [a desired product] fits their style and aesthetic. » But the much bigger news is that it also makes sure products fit in the space intended — and it doesn’t need a depth-sensing phone to do that. Sorry, iPhone X.
From one perspective, there is little new here in this app; the ability to take potential purchases and superimpose them on real-world images has been around for years, with home improvement chains taking the lead. But what Amazon is claiming goes well beyond what others have done, and it does it in an environment (the Amazon app) where the shopper is already comfortable.
The hardest part of this AR effort is not getting precise dimensions of the object being sold — presumably, the supplier has already delivered that — but instead is determining the precise measurements of everything else in the room. That’s the only way to determine if a refrigerator will truly fit in the intended space and whether it will work when its bottom drawer is fully extended.
From an email exchange with an Amazon manager, who asked that her name not be used: « AR view uses Apple’s ARKit. ARKit gives us the scale of the room or surface. It can detect surfaces and how large those surfaces are. Our models are dimension accurate and precise. The 3D capabilities of phones have no bearing on this feature. »
This is where things get a bit tricky. The Apple ARKit referenced, based on how Apple describes it on its site, clearly envisions the iPhone X and its TrueDepth camera. That said, it still works decently without a TrueDepth camera.
Apple said the ARKit runs on the Apple A9, A10 and A11 processors. To put that into a phone-model context, it would exclude the iPhone 6 (which uses the A8) and anything older. But it would include the iPhone 7 (which uses the A10) and anything more recent, as well as current iPads (which use the A9).
« ARKit is an umbrella SDK, which provides a variety of AR features across the iOS product line. TrueDepth is just one piece of that. TrueDepth is not required for our feature, AR view, » the Amazon representative said in an email reply.
Amazon used Apple’s ARKit to create an augmented reality feature in its shopping app that works on phones other than iPhone X.
That is true, as I tested the new capabilities on an iPhone 7 Plus, and it worked decently. By the way, this may be unrealistic, but adding in a sense of weight of the product and likely weight-bearing strengths of objects in the room would be a nice addition. I placed a large couch on a flimsy cardtable in my office and it held fine. Yeah, not likely.
But it’s dimensions and room context were impressively accurate. Apple offered this explanation for how it delivers its AR magic:
« With ARKit, iPhone and iPad can analyze the scene presented by the camera view and find horizontal planes in the room. ARKit can detect horizontal planes like tables and floors, and can track and place objects on smaller feature points as well. ARKit also makes use of the camera sensor to estimate the total amount of light available in a scene and applies the correct amount of lighting to virtual objects. »
The key advantage in all this is that it gives shoppers a very strong reason to explore home purchases from within the Amazon app rather than rivals. Given that Amazon is typically price-competitive and it makes purchasing quite easy, this could be a very nice revenue boost during the holidays and beyond.
Still, Amazon could have done far more. When it reveals that the selected item is too large for the selected space, why doesn’t it immediately offer versions of that product that WOULD fit? Isn’t that the logical next step? It could have used artificial intelligence (AI) to offer recommendations. Things such as “I think it clashes with your current design. Try choosing it two shades lighter. Click here to see that in AR” or “That item seems too large. Would suggest using something not larger than nine inches high. Click here to several options that I think might fit better.”
Or instead of using pure AI, it could have used crowdsourcing and a little AI to determine the most popular and second-most-popular answers. Or it could have offered a Home Depot-like expert service, where interior decorators react to item choices with the desired room.
The goal here is to make Amazon’s app the ultimate shopping companion. AR is a great step, but why stop there?
