Vision Pro can make any surface appear to be a control, or an app display
Developers using Apple Vision Pro have learned they will be able to create controls and displays, and have them appear to be on any surface in the user's room.
Apple has been working on Vision Pro for a long time, and along the way it has applied for countless patents that were to do with it — even if it wasn't always obvious what the company's plans were. Now one intriguing patent application from 2020 has been revealed to be a part of Vision Pro that will help developers.
Developer Steve Troughton Smith has discovered that it's possible to pick a surface within the headset's field of view, and then place any app so that it appears to be actually on that surface.
"A natural way for humans to interact with (real) objects is to touch them with their hands," said Apple's 2020 patent application. "[Screens] that allow detecting and localizing touches on their surface are commonly known as touch screens and are nowadays common part of, e.g., smartphones and tablet computers."
Troughton Smith used an Apple Music app for his experimentation, but it can be any app and seemingly any controls. So where Apple's virtual keyboard for the Vision Pro is not practical for long typing sessions, it could be that a user's desk is turned into a keyboard.
That would still make for a stunted typing experience, as there would be no travel on the keys. But there are already keyboard projectors that use lasers to display a QWERTY layout directly onto a flat surface.
