3D's New Dance: The Evolving World of Spatial Media
3D's New Dance: The Evolving World of Spatial Media
Apple Vision Pro, Immersive 3D Content, and NeRFs are Back?! Plus, Midjourney drops Niji v6.
šŖĀ Creative Technology DigestĀ
Your weekly dose on the future of creation & computing.
š Video Essay: Making NeRFs Practical Again
Hello Creative Technologists! Did you order Apple Vision Pro? I did and will receive it on Feb 2nd. Canāt wait to try it and share more.
With all these āspatial computingā headsets hitting the market, we are going to need spatial content. And guess what? 3D scanning changed again, making NeRFs competitive with Gaussian Splatting. Needless to say, weāll continue to see a dance in 3D formats. Meanwhile, hereās everything you need to know about the latest hotness ā SMERF and Nuvo:
šĀ 3D Scanning Changed Again. NeRFs are SO back!Ā
Topics covered:
00:00 Wait, NeRFs are back?!
00:54 SMERF vs. 3DGS
02:30 How SMERFs work
03:40 Editing NeRFs with Nuvo
05:40 3D: What To Expect In 2024?
š„½ Apple Vision Pro: Spatial Computing Experts Have Entered The Chatā¦
Roughly 200K or so early adopters (like myself) are sliding into the Vision Pro party. In other words: Metaverse is out, and Spatial Computing is in.
Brb updating bio to spatial computing expert
Seamless integration with the Apple ecosystem is amazing, but Appleās real advantage? The power of their hardware and dual-chip architecture.
Apple's prowess in compute and hardware gives them a tremendous edge in the race to build the ultimate mixed reality headset.
As reviewers are noting, Apple Vision Pro is now the gold standard, and it has a lot to do with their dual-chip architecture āļø
(1/5) Thread š§µ
Yaāll know I love 360/VR media. But when we reimagine the consumption experience from the ground up, then things get really fun š
In an Apple Vision Pro world, the viewing experience for telemetry-rich sports like Formula 1 can transform dramatically šļøšØ
We must resist the urge to put floating windows everywhere and think spatial-first. @JohnnyMotion makes this point beautifully ā¬ļø
However, can we view everything? It seems big media apps are missing the party ā or at least playing a āwait and watchā game before supporting yet another platform.
Apple Vision Pro pre-orders are opening up, but there are some noticeable absences from the 3rd party lineup ā Netflix, YouTube and Spotify to name a few.
Get ready forā¦ Overlapping metaverses :)
In a way with ARKit vs ARCore it already is the case. Now as far as the Questā¦ twitter.com/i/web/status/1ā¦
š„ Creation Corner:Ā
1. Open Source 3D Capture - A big week for the 3D Capture community! Nerf Studio Team brings a commercially-ready Gaussian Splatting implementation. Google AI releases code for Zip-NeRF and CamP. Hereās why itās a big deal:
Big day for open source 3D capture! š
@nerfstudioteam hits 1.0 and gets a *commercial-ready* Gaussian Splatting implementation
@GoogleAI open sourced Zip-NeRF ā the highest quality 3D reconstruction pipeline, setting the stage for a future SMERF release
2. TikTok Research: Depth Anything - Depth estimation is critical for both real-time and off-line workflows in AI art, AR, and real-world analysis. And Bytedance? Well, theyāre bringing a foundation model to the party:
Donāt sleep on ByteDance research. Their new AI model produces insanely crispy depth maps.
TL;DR Better relative depth than MiDAS. Better metric depth than ZoeDepth. And get ready for a better depth-condition ControlNet š
Oh and they already shipped it in TikTok too! (1/3) š§µ
3. Midjourney for Anime - Niji v6 for Midjourney dropped, and it's fire š„
Oh yeah! Niji v6 for Midjourney just dropped, and it's pretty fire š„
My initial generations below. Can't wait to see some banger anime content in our feeds this week! šæ
š± Long-form Listening:Ā
1. Twitter FrAIday Recaps: Vision Proās Impact, GPT advancements and AI in Marketing Strategies
Every Friday a group of us get together on X to discuss the latest updates in AI. The last two weeks, we discussed the launch of Apple Vision Pro, its focus on resolution displays and immersive audio and potential to revolutionize spatial computing. We also discussed the monetization of ChatGPT, creative workflows and the impact of AI on art and creativity. Watch the full episodes posted by Heather Cooper on YouTube.
š Stay in Touch:Ā
Got feedback, questions, or just want to chat? Reply to this email or catch me on social media.