Microsoft is filling out its set of mixed reality tools with a real-time collaboration platform called Microsoft Mesh. Officials showed off demos and futuristic scenarios involving Mesh on March 2, the first day of its virual Ignite 2021 spring conference.
Microsoft execs described Microsoft Mesh as a “new mixed reality platform powered by Azure” that is built to create a sense of presence on a variety of devices. Microsoft Mesh will enable users to view and interact with mixed reality content together in a single virtual or actual space. Mesh builds on work Microsoft already has done around Azure Spatial Anchors, Azure Object Anchors, Azure Rendering, the AltspaceVR social virtual-reality technology Microsoft bought in 2017 and other internal mixed-reality research and commercial efforts.
While Microsoft showed off holoportation as part of today’s Ignite demos, holoportation capabilities are not currently part of the publicly available Microsoft Mesh platform. Holoportation allows individuals with mixed-reality devices, including the HoloLens, to see and interact with remote participants in 3D as if they were present in their physical space. Microsoft researchers have been working on Holoportation in various incarnations, over the past several years.
With Mesh users will have “the richest experiences” in mixed reality or virtual reality. But Mesh will work with Microsoft’s HoloLens 2, a range of virtual reality headsets, smartphones, tablets and PCs, officials said. Initially, via Mesh, people will appear as avatars in shared virtual experiences. “over time” they will be able to use holoportation to project themselves.
If these scenarios sound familiar to Microsoft users and watchers, there’s a reason. Microsoft has been talking up this vision of how mixed reality could change collaboration since five years ago. (Remember this Microsoft futures video, showing a mix of people appearing as avatars and holograms working together to design a store from 2016?)
Microsoft officials said the work the company has done over time in areas like hand and eye tracking and creating persistent holograms and AI models that can create “expressive” avatars is what is enabling the company to finally deliver on this long-time vision. Microsoft Mesh was built on Azure and uses various Azure data, AI and mixed reality services and store holographic content in Azure.
Here’s Microsoft’s architectural diagram of the Microsoft Mesh platform:
“In the coming months,” Microsoft plans to offer developers a suite of AI-powered tools for tasks like creating avatars, session management, spatial rendering and sync across multiple users. Customers already can download the first two apps built on the Mesh platform: The Microsoft Mesh app for HoloLens, which allows team members to remotely collaborate; and an updated version of AltspaceVR, which is “Mesh-enabled.” AltspaceVR users can log in with Microsoft-authenticated credentials to host meetings and work gatherings in virtual reality, officials said. Microsoft is encouraging devs inside and outside the company to Mesh-enable other apps and integrate them with Teams and Dynamics 365.
The holoportation piece of Mesh will likely take a bit longer, even though Microsoft has been working on holoportation and telepresence technologies for years and demonstrated on stage at a 2019 partner event holoportation and real-time translation working in tandem.
In 2016, a number of those working on holoportation at Microsoft and Microsoft Research quit and went to start a company called perceptiveIO Inc., which seemingly was bought by Google.
