I always find it a tad disconcerting when an article I carefully planned out doesn’t end up anything like the plan. It doesn’t happen often, but sometimes an article can take on a life of its own.
This was one such article. The assignment was simple. In honor of this week’s quiet VisionOS 2 release, I was to write about my view of the best VisionOS 2 features available now (and what’s delayed).
Fundamentally, I was disappointed. The new hand gestures don’t appear to work if you, like me, have some accessibility features turned on.
The ultrawide Mac virtual display, which I still expect to be the killer app for the Vision Pro, isn’t available until “later this year.”
Also: The best VR headsets of 2024
Sure, there’s now mouse support. The new Bora Bora environment is nice. And you can rearrange the icons on your home view. But, yawn.
A stupid gimmick (or so I thought)
And then, I was going to have to mention the dumbest new feature of them all, a completely stupid gimmick. That’s the carnival sideshow feature where you can create spatial images from your 2D photos.
Except, well, that stupid gimmick is make-me-cry awesome. Seriously.
I’m struggling to write about this because this feature is just about the most unessential use of advanced technology to ever exist. And it’s certainly not a justification for buying a $3,500 headset.
Also: Meta Quest 3 review: The VR headset most people should buy in 2024
But… after experimenting with it yesterday and today, it may also be the first thing you want to show someone if you’re trying to show off your painfully overpriced headbrick.
I mean, this is some Marty McFly Darth Vader from the planet Vulcan, Ghost of Christmas past, mind-bending bonkers stuff.
It’s a 30-second theme park ride that can slam you 20 years in the past, rip out your guts, and drop you head first into a surrealistic uncanny valley of your own mind.
I am not exaggerating. This is not hyperbole. Once you strap your headset to your head, launch the VisionOS 2.0 Photos app, and click the little magic button, be prepared for a digital acid trip that Ken Kesey and the Merry Pranksters would be proud of.
Okay, let’s back up a bit and put this into context. VisionOS 2.0 includes a feature in the Photos app that uses AI and machine learning to transform a flat photo into something that appears fully 3D.
Also: Stanford’s VR breakthrough could spell the end of clunky headsets – thanks to AI
Unfortunately, nothing I could screenshot and show you here will do justice to the experience. In fact, the only way you’ll be able to fully grok what Apple has wrought is to put on a Vision Pro and try it out.
What I can do is try to describe the process of what happens and how I felt after following the white rabbit down the rabbit hole.
All of this takes place inside of the updated Photos app. Once you open an image, you’ll see a little 3D cube button, like this:
When you press the button, a light shadow flows across the image, the subject is highlighted, and in about five seconds, a 3D version of the image is created.
I want to be clear here. This isn’t a 3D scan that you could use to recreate the subject on a 3D printer. All it does is slightly alter the pixels so if you move your head, you see a sense of depth.
But that description doesn’t do the results justice. When you have the right subject, pressing the little button makes what you’re looking at go from a photo to real. I’m not even sure how to properly describe it, because the effect isn’t just 3D. It’s almost a realifier button. It somehow manifests the subject of the image so that you’re suddenly looking at the real thing.
When you tap the realified image, it expands into the full environment, surrounded by a ghostly haze. I only noticed one instance of artifacting, with some streaks showing under my pup Pixel’s right ear.
When looking at this image through the Vision Pro, it looks like I’m really looking at Pixel. But because the image isn’t moving, it’s almost like looking at an animal in a taxidermy shop. That wasn’t too bad, because the pup was sitting on my lap while using the device.
But then I got the idea that I wanted to see Sammy again. Samantha was my cat. She lived in the late 1980s and 1990s and passed away in 2001. I adored her and haven’t been able to bring myself to look at the few snapshots I have of her because I missed her so much.
Also: We’ve used every iPhone 16 model and here’s our best buying advice for 2024
So, in some ways, this was a very baaad idea. It hit me right in the feels.
But I wanted to see how an old film print would respond to the 3D-ification process, or if it would even work. I dug out a few scans I’d previously made of my film negatives and imported them into my Photos app.
And then I opened up the pictures on the Vision Pro and realified them.
Did you know that if you cry inside the Vision Pro, your tears just build up inside? There’s not really room for the liquid to leak out.
She wasn’t moving, but she was real. She was there. And, in the Vision Pro, she also appeared to be the size of a couch.
Review: Apple Vision Pro: Fascinating, flawed, and needs to fix 5 things
It felt like I could reach out and touch her. It was a very trippy experience. I miss my little girl, but I didn’t expect that level of reaction.
I looked at three or four more pictures of her, all made ghostly by the expanded effect, and it took me down. Big, strong, bearded guy tearing up over a kitty he hasn’t seen in more than two decades. It’s embarrassing. But that’s the unexpected power of this feature.
What works and what doesn’t
I found a couple of pictures of me going back a bunch of years. While the 3D effect was there, photos of me as a grownup weren’t terribly impressive. I also tried a number of pictures that had strong subjects, like a close up of an F-14 Tomcat, a tugboat, and a helicopter.
In addition, I tried some scenic views to see how they would perform. All of them had the cheap feeling you get from one of those old View-Master disks. They weren’t all that special.
Also: My biggest regret with upgrading my iPhone to iOS 18 (and I’m not alone)
But photos with very clearly defined small subjects, like pets and toys and stuffed animals, work extremely well. I imagine pictures of babies and possibly of small children might also stand out.
In fact, this image of little me did pretty well. The original scan was only 800×600. It was too small for the 3D conversion, so the little cube button was missing. Actually, I wasn’t sure if the button was missing because the image was black and white, or too small. To test, I brought the image into Photoshop and used Photoshop’s smart resizing features to scale it up 4X.
That worked. It wasn’t as real-feeling, but you could see the “me” part of the image was in front of the house in the background (which, if I remember correctly, was my friend Jonathan’s).
After a bit of testing, it’s clear that this feature works on any image (black and white or color) that has enough pixel density to give it something to work with.
A surprisingly powerful impact
Overall, the VisionOS 2.0 update is a nice little update. The killer productivity feature, the ultrawide virtual desktop, isn’t available yet, which reduces the update’s value considerably. Still, anything that improves on the original VisionOS is welcome.
But the standout feature was definitely the ability to convert existing images to 3D. I really didn’t have any respect for the idea. But after I tried it, I found it to be one of the most impressive showcases of what is or might be possible with immersive VR.
Also: How VR and digital twins are streamlining remote work in technical professions
I do seriously recommend showing off the 2D-to-3D conversion feature to new people who are Vision Pro-curious. Seeing someone else’s 3D immersive experience is one thing. That is impressive. But seeing your long-gone, much-beloved kitty appear as if she were real right in front of your face had a visceral impact that demo content can’t touch.
It’s not a feature anyone would use regularly. I’m not even sure it would be emotionally healthy to use it regularly. But it sure does show the impact and power of immersive 3D unlike just about anything else I’ve tried. That’s because it takes as something old and familiar and personal as an old photo and breathes life into it. Damned if that didn’t feel just a bit like magic.
But it also came at a price. Seeing Sammy there had an emotional impact I just didn’t expect. I’m still shaking it off. I’m really glad I had the chance to experience it, but it’s not something I want to try again anytime soon.
Also: Who’s afraid of VR? I was – until I tried Meta Quest 3
What do you think? Do you have a Vision Pro? Have you upgraded to 2.0 yet? Have you tried any of the new features? Let us know in the comments below.
You can follow my day-to-day project updates on social media. Be sure to subscribe to my weekly update newsletter, and follow me on Twitter/X at @DavidGewirtz, on Facebook at Facebook.com/DavidGewirtz, on Instagram at Instagram.com/DavidGewirtz, and on YouTube at YouTube.com/DavidGewirtzTV.
