What if you could use the augmented reality features of a LiDAR-equipped iPhone to see how a room might look with all its furniture removed? That’s the idea behind a new AR prototype shown off by Shopify’s Russ Maschmeyer on Twitter. The demo, which he refers to as a “reset button” for a room, puts Ikea’s virtual design tool to shame. But it’s just an internal experiment for now that offers an interesting peek at the kinds of experiences that could be possible with Apple’s latest AR APIs.
In a video, Maschmeyer shows how the prototype is able to seamlessly erase a room’s contents, allowing the iPhone to be used to peer around the empty space in AR. In subsequent tweets, Maschmeyer explains how the technology might one day be useful for e-commerce sites, allowing them to show customers how new furniture could look at home without existing furniture getting in the way.
Lots of interest in this “reset button” for your room we built @Shopify that uses @apple’s new #RoomPlan API. Here’s how we built it… 1/12 pic.twitter.com/YdNP1vebVW
— Russ Maschmeyer (@StrangeNative) July 13, 2022
The prototype is built with Apple’s RoomPlan API, a tool for developers that the company detailed at WWDC this year. It’s designed to let an iPhone or iPad equipped with a LiDAR sensor scan a room, understand its geometry and furniture, and build a 3D model that app developers can use however they want.
Maschmeyer’s thread is an interesting look at what Apple’s technology is like to use in practice. For example, he explains how the model generated from the scan is completely untextured, so Shopify’s prototype needs to gather texture data from the phone’s camera, and then work out how to extend those textures behind the furniture that might currently be obscuring parts of the walls and floors. It then overlays the resulting 3D model onto the real room.
We can now use inpainting techniques to fill in the redacted texture. We were pleasantly surprised by the results produced by off-the-shelf tools like Photoshop and https://t.co/vMbvV0Fa92. [11/12] pic.twitter.com/jGOdZ7it0b
— Russ Maschmeyer (@StrangeNative) July 13, 2022
The results, it has to be said, are a lot more impressive than what I experienced when I tried out a similar virtual design tool from Ikea. Although Ikea’s app is also able to erase furniture from a room, it’s only able to present the results as a 2D image. In contrast, Shopify’s prototype seems to allow you to continue looking around the (now empty) room in augmented reality. But, in fairness to Ikea, its approach doesn’t require a LiDAR-equipped iPhone (any iPhone will do). It’s also a real piece of technology that people can download and use in their own homes, rather than an internal tech demo.
Apple has been showing off its AR tools at developer conference for years, but I think it’s fair to say that we’ve yet to see a killer app for the technology emerge (I don’t count Pokémon Go because of how so many people play without the AR mode enabled). But prototypes like these make me hopeful that interesting uses for the technology could be just around the corner. Like, maybe January when Apple’s AR / VR headset is rumored for release.
