Era of Spatial Computing

Minority Report is coming to the retinas on you

On 5 June 2023, Apple announced Vision Pro at WWDC. So what?

[Image credit from Apple official site: Vision Pro]

The iPhone was not the first smartphone, the iPod was not the first portable music player, and the iMac was not the first personal computer (PC). Vision Pro is not the first Virtual Reality (VR), Mixed Reality, or Augmented Reality (AR) headset. Apple is well known for its late mover strategy by perfecting early experimental conceptual technologies. We have seen Apple do it successfully and repeatedly:

• User Graphical Interface and mouse for computer, solving the hard-to-use DOS command on early Windows and mainframe terminal.

• The iPhone's touch finger-navigation interface replaced the clumsy arrow navigation buttons on smartphones or the physical keyboard on Blackberry.

• Subscription-based Apple Music on iPod has transformed the music industry and disrupted the music distribution channels. It even successfully cleaned up the practice of music file-sharing piracy. Apple Music inspires new players like Spotify, and such disruption totally put my CD collections in the donation bin.

• Despite not being the cheapest tablet, the iPad still dominates the tablet industry by providing a complete ecosystem of hardware and software. iOS apps have the most comprehensive selection, and the OS and hardware efficiency remain the top dogs in the market.

Like other groundbreaking product innovations by Apple, Vision Pro is opening up a new era of wearable computing and visualization devices - Spatial Reality.

• Similar to the “Minority Report (2002)” movie, the kind of visualization method portrayed by Tom Cruise. The difference was that Tom Cruise used a wearable globe to control the projected analytics on screens, and the environment scanned iris to identify individuality. Vision Pro has built-in computing and can navigate without any external controlling device, which is a first.

  (Here is a link to recall your memory about the interactive user interfaces shown in the movie Minority Report). [Image credit from Tom Mendelsohn’s article about Ultrahaptics].

  

• Vision Pro is a new breed of gadget, unlike any existing product. It is a mobile computer, a projection screen, a sensory input controller, speakers, virtual reality, and augmented reality. A new category of device that competes with technology gadgets in multiple categories:

  • Simulation games and training devices

  • 100” TV or short-throw laser projector. Of course, Vision Pro cannot be shared with others like an active projection screen. It is equipped with 2 tiny but 4K high-fidelity displays.

  • Vision Pro is a computing device for users with special needs or specific disabilities. Equipped with an M2 processor and R1 visual processing, it is more powerful than the majority of computers that we use today.

  • 3D and 360-degree video recording devices, like the Insta360 camera with 2 cameras. Vision Pro has 12 cameras. How many gadgets do you know to have 12 cameras?

  • Spatial audio, like the Sony Neckband Speaker. Of course, the tiny speaker cannot compete with the more prominent speaker.

  • A new level of gesture, vision, and voice control. Vision gesture and voice control are not new; many camera devices can detect the user’s voice and gesture commands, but Vision Pro has up the game and set the bar for new market expectations. Vision Pro has 5 sensors that monitor the user’s hand gestures at all times and cameras that monitor our eyes and follow our focal point.

• Contents are integrated and seamlessly transitioned among compatible devices (smart cars, smart homes, smart devices, etc.). Huawei’s Harmony OS, Samsung, and Alphabet work on ecosystems like Apple. This is a competitive platform that is different from the individual gadget. We have seen how Harmony OS supports a seamless transition from computer to mobile to intelligent display inside a car while having an uninterrupted online meeting.

• The mixed reality wearable market is heating up. We will see a new generation of wearables from Meta (which is releasing Quest 3), Microsoft (Hololens 2), and Magic Leap 2 but they are still stuck in industrial-used AR headsets. Google continues to work on its AR Glasses which is promising (personally I like the AR enhancements on Google Maps).

Apple has been postponing Vision Pro's release in the past few years. Apple is well known for its product design, so I am amazed at the thoughtfulness and readiness of the design thinking behind Vision Pro.

• Stakeholders consideration. Vision Pro has not only been designed for various user personas but has also taken the people nearby as stakeholders. Visio Pro has many designs allowing users to interact with others or situations when wearing the Vision Pro.

  • It portrays the user’s eye on an external screen as if they are wearing transparent goggles. Eye contact is important for human (or even pet) interaction. The user sees the external world via the camera feed (like our car's rear camera projection on the LCD screen), but external users see the user’s eyes on a screen.

  • When wearing Vision Pro for online meetings, other participants can see your avatar. Thanks to the cameras that analyze your hand's movement, the avatar can move based on your real-life gestures and expressions.

• Use cases that Vision Pro covers are exciting (meaning it solves real problems for many personas).

  • The individual who likes to be immersed in personal entertainment, from 2D to 3D, in the living room or on a plane

  • The parent who wants to capture and re-live the precious family moments

  • Information workers who like to engage in works with more immersive screens and larger real estate, who also do not want to be desk-bound

  • Users who cannot operate using a mouse or touchpad and wish to use eye plus hand gestures to control the computer, who like to see the display without any physical monitor screen (e.g. while preparing a meal in the kitchen)

  • A kid experiencing any computing device for the first time can quickly learn navigation, control, etc.

What I would like to see in the coming years are:

• The price of Vision Pro devices may drop to the USD thousand plus range, which is more affordable for the general crowd. The current release price of Vision Pro is USD 3,499 in early 2024.

• Accessories or add-on functions such as a haptic feedback glove, shirt or brainwave reader (like Muse) complement the user experience.

• Wireless charging and longer battery life. Vision Pro needs an external power bank with a wired connection, which lasts only 2 hours. We have seen pilot experiments in the US and Europe, where special road pavements can charge traveling electrical vehicles (EVs) wirelessly.

• We need a variety of devices with different form factors, perhaps a Google Glass-like wearable that is usable in more situations. Vision Pro is still an immersive device, and most users may still feel awkward wearing it in social settings where we are expected to have more normal social interaction with other people present in the same space.

• In new simulation apps like online fashion retail, consumers can test a new cloth virtually with a mirror or avatar or visualize the renovation design of their house. The gaming and adult entertainment industries will likely be early adopters of such new technology.

This year, generative AI and spatial computing will significantly influence the new computing experience in the coming years. Minority Report 2002 is here in 2023. While we are excited about the possibilities of more convenience and immersion in the new technologies, we shall be more aware of whether we consume more passive content feeds or use the technologies to grow by deliberate learning. As an individual, am I consuming more entertainment or creating more contributions?

---