Facebook is expected to release its first smart glasses product later this year, but the company also has an eye on your wrist for creating a proper augmented reality (AR) experience – and it’s one that doesn’t involve an iPhone.
Apple, Amazon, Microsoft, and Google all have widely available hardware, but Facebook’s Portal didn’t hit its stride until the pandemic, unlike existing hits such as Amazon’s Echo smart speakers, Microsoft’s Surface laptops, Google’s Pixel phones, and the iPhone.
Facebook does have Oculus virtual reality headsets, though – and now the social networking company is exploring the idea of glasses synced-up with a wrist-borne device.
SEE: Virtual hiring tips for job seekers and recruiters (free PDF) (TechRepublic)
According to Facebook, you’d be less distracted than smartphone users by this future technology, which exploits the company’s advances in artificial intelligence (AI) fields such as computer vision and natural language processing.
“Imagine a world where a lightweight, stylish pair of glasses could replace your need for a computer or smartphone,” the company says in a new blogpost.
“You’d have the ability to feel physically present with friends and family – no matter where in the world they happened to be – and contextually-aware AI to help you navigate the world around you, as well as rich 3D virtual information within arm’s reach.
“Best of all, they’d let you look up and stay present in the world around you rather than pulling your attention away to the periphery in the palm of your hand. This is a device that wouldn’t force you to choose between the real world and the digital world.”
Facebook wants to get even more personal with users than Apple Watch, billed by CEO Tim Cook as Apple’s most personal device ever. Instead of tracking heart rates and movement, Facebook wants a direct connection to the brain that displaces the mouse, trackpad or touchscreen. That kind of hardware would help to remove its dependence on Apple, which keeps introducing privacy changes that harm Facebook’s business model.
Facebook is aiming for “ultra-low-friction input” to create a shortcut from thought to action. In 2019, it bought CTRL-Labs – a US startup that uses electrical muscle signals to control a virtual hand – and is exploring sensors that connect to the brain and spinal cord as an input for AR glasses.
“You might gesture with your hand, make voice commands, or select items from a menu by looking at them — actions enabled by hand-tracking cameras, a microphone array, and eye-tracking technology,” Facebook explains.
“But ultimately, you’ll need a more natural, unobtrusive way of controlling your AR glasses. We’ve explored a range of neural input options, including electromyography (EMG). While several directions have potential, wrist-based EMG is the most promising. This approach uses electrical signals that travel from the spinal cord to the hand, in order to control the functions of a device based on signal decoding at the wrist.
“The signals through the wrist are so clear that EMG can detect finger motion of just a millimeter. That means input can be effortless – as effortless as clicking a virtual, always-available button – and ultimately it may even be possible to sense just the intention to move a finger.”
SEE: How 3D printing is helping CERN scientists upgrade the world’s largest machine
Facebook also believes its hardware should be created as “soft, all-day wearable systems” that can extract data directly from the skin’s surface and use AI algorithms to infer intent from tiny finger movements.
“AR glasses interaction will ultimately benefit from a novel integration of multiple new and/or improved technologies, including neural input, hand tracking and gesture recognition, voice recognition, computer vision, and several new input technologies like IMU finger-click and self-touch detection,” Facebook says.
