Apple Vision Pro - Everything You Need To Know
Apple Vision Pro - Everything You Need To Know
Step aside AR/VR, spatial computing has arrived and it's 20% here - and it's really good but also heavy and incomplete. Let's get into it.
🪄 Creative Technology Digest
Distilled insights and experiments on all things AI, 3D/VFX and Spatial Computing.
In this edition:
🌶️ AVP hot take: Apple Vision Pro is extraordinary but frustratingly closed off
😎Vision Pro highlights: including uncanny Apple Persona avatars, immersive 6DoF petting zoos, and T-Pain + Gen Z reactions
🧠AVP thought starters: Vision Pro provides superhuman glimpses, and sorry Usher but you're now a boomer apparently!
🎓Deep dive: How to create immersive content specifically for the Apple Vision Pro headset
🍇 Distilled Analysis: 2 Weeks With The Apple Vision Pro: Trapped In A Beautiful Walled Garden
I’ve used the Apple Vision Pro for 2 weeks now and here are my unfiltered thoughts — you might even call it a hot take 🌶️ 😅
Overall: I’m blown away, absolutely hyped… but also? Frustrated. Why is Apple making it SO HARD to tap into the existing VR media scene?
There is a plethora of VR 180 and 360 content out there. And they’ve got this sweet immersive video player tucked away in Apple TV, with that signature Apple polish. Great I can watch an Alicia Keys video. YouTubeVR has millions of more videos I can’t currently watch.
So I try to drop manually converted spatial videos directly into the Mac Photos app — nope! Gotta jump through iCloud hoops to retain the metadata? It’s ridiculous. Almost feels like Apple wants to gate keep ALL immersive content on the headset. Ok fine but you can shoot stuff on iPhone spatial videos or on the headset itself. Awesome, but to edit it you need to use a 3rd party converter to turn it back into well adopted VR media formats to edit in Adobe or Resolve!
There are many killer ARKit apps on the App Store, it might’ve made sense to get some of those ported over! Missed layup IMO. And don’t even get me started on the broken webXR support. 3D websites like Luma AI and Polycam should’ve been immersive 3d on day 1. It feels like they just shipped an unfinished product. Which might be the case, considering how much of that mind-blowing WWDC 2023 stuff is STILL missing. Remember those awesome shared SharePlay experiences they demoed? Poof, gone. Even simple stuff, like shared spatial anchors that ARKit already supports for multiplayer AR, is nowhere to be found.
I mean they can pass through reality with imperceptible latency, but I can’t have shared experiences with users who also have Apple Vision Pro in the room, and I am relegated to iPads on screens? Not much better than zoom and a massive underutilization of what this hardware is capable of in terms of realistic co-presence and remote collaboration.
Look, maybe this is all growing pains. Maybe they shipped with 20% of the roadmap ready, and we’ll just have to wait. But right now, the Apple Vision Pro feels like this super shiny walled garden in the middle of a sprawling VR playground. Here’s hoping they open the gates soon… 🔐
💬 Join the lively conversation on Twitter/X and share your thoughts
🎥 Vision Pro Highlights - The Culture Is Changing, Fast!
1. Apple Persona Avatars: Uncanny but cool 🔗 [Twitter/X Link]
Apple’s telepresence is uncanny but cool — this’ll be a boon for remote collaboration.
But I’m baffled that Apple didn’t roll out SharePlay as a flagship feature.
Like, I want to watch content with my Vision Pro homies chilling in a 3D environment — and we can’t.
Given… twitter.com/i/web/status/1…
I imagine this is what support calls will feel like in the near future.
Remote experts will be able to “telestrate” over our field of view and annotate things spatially.
Eventually it’ll be a multimodal AI doing the annotation. But for now, humans will be far more effective.
2. Immersive 6DoF Content: Petting dinos in your living room [Twitter/X Link]
Tryna get dinosaurs to calm down in your living room is wild experience y’all 🤯
Apple got me feeling like Chris Pratt over here!
I've tried almost every VR headset over the past 10 years, but Vision Pro is by far the most immersive.
Exploring volumetric 3D captures inside Vision Pro is gonna be wild
Can’t wait to explore all the spaces, places and moments I’ve captured over the years
Polycam app with native support tomorrow. @RadianceFields says it feels like you’re actually there
3. T-Pain 💗's AVP, Gen-Z Agrees: Cyberpunk much? 🔗 [Twitter/X Link]
T-Pain is all in on Apple Vision Pro — says resistance is futile!
We had a chance to end up with google glass with the “dragon ball z thing on the side of it”
But look what we’re left with now… I guess full-blown immersion in a Marriott lobby? Lol
Gen Z going wild with the Apple Vision Pro. Bro literally transformed into a stormtrooper… or a member of Daft Punk 💀 😭
Video credit: justinwiebvr
Apple sure knows how to do launch day!
On second thought, I should’ve gone with in-store pickup :)
#AppleVisionPro
4. 3D Content + AVP = Heaven: Infinite spatial canvas 🔗 [Twitter/X Link]
Flipping between real & virtual worlds with my 3D scan in Vision Pro is pure eye candy 😻
The environmental lighting change reveals all the high frequency detail baked into the normal maps 🌅
Used Sketchfab to convert my photogrammetry model to USDZ, then you can open it… twitter.com/i/web/status/1…
Nothing like soaking in a “view” at the end of a bike ride
Blending the real and virtual has tons of potential
#AppleVisionPro
If James Cameron was “really, really blown away” by the Vision Pro and had a religious experience, then I can’t wait get mine tomorrow
He’s been on the cutting edge of virtual production for decades. Hard person to impress.
5. Apple Went With Style > Function: Look good but heavy so 3P mods galore 🔗 [Twitter/X Link]
This might actually be the comfiest Vision Pro mod I’ve seen, and yet it goes so hard
🌱 Thought Starters
1. Vision Pro provides a glimpse of superhuman vision. 🔗 [Twitter/X Post]
Imagine having this in a Tesla with the Apple Vision Pro. You're visualizing a crystal-clear fusion of the sensor streams from the car—effectively allowing you to "see through" the car and reveal a 4D world that you can explore, interact with, and get stuff done in the real… twitter.com/i/web/status/1…
2. Usher is now a boomer, apparently - Sorry Usher! 🔗 [Twitter/X Link]
On behalf of all millennials
🎓TL;DR Deep Dive: Creating Content For Apple Vision Pro 🔗[Twitter/X]
Apple Vision Pro hype has made immersive media cool again. But how do we capture such 3D content?
1. On the pro end, I expect a resurgence in expensive multi-camera capture systems. Not just for stereoscopic video, but also volumetric content. If you haven't already gone down… twitter.com/i/web/status/1…
💌 Stay in Touch:
That’s it for this edition. Next one we’ll be going deep on Sora and Gemini mania. Got feedback, questions, or just want to chat? Drop a reply to this email or DM me on social media.
Cheers,
Bilawal Sidhu