Pokémon Go Built a Secret 3D Map of the World. So Did Everyone Else.
Every layer of the physical world is already being mapped. Most people are part of the dataset and don't know it.
People are understandably a little alarmed at having unwittingly helped Niantic build a 3D map of the world — 30 billion crowdsourced images across twenty million locations, now being licensed to train delivery robots.
I actually sat down with the guy running this — Brian McClendon, SVP of Engineering at Niantic Spatial, the man who co-created Google Earth — on The TED AI Show last year. What he told me paints a very different picture than what the headlines are saying. I made a full video breaking this down, plus the dozen other companies quietly mapping the world at every layer, from your doorstep to orbit.
Pokéstop data isn’t what you think it is
The MIT Technology Review story broke that Niantic had licensed its massive spatial dataset to Coco, a delivery robot company, and John Hanke — CEO of Niantic Spatial — said something that set the internet on fire:
“It turns out that getting Pikachu to realistically run around and getting Coco’s robot to safely and accurately move through the world is actually the same problem.”
Reddit and X threads blew up. Some calling it a betrayal of trust. Others saying you agreed to the terms of service. But here’s what most people missed.
Pokéstop data isn’t what you think it is. It’s not a surveillance panopticon of your neighborhood. These are static captures of parks, statues, murals, landmarks — the places people congregate. Brian described it as “building the map from the bottom up, from the locations where people spend time.”
Think of those twenty million waypoints as basically the inverse of what Google mapped with Street View. Google mapped the drivable streets. Niantic mapped where people actually hang out. Cool data, genuinely useful for visual positioning — but very different from what the headlines imply.
So unless you’re concerned about a panopticon of all the fountains in the world, I wouldn’t be too worried.
From Pokémon to centimeter-accurate positioning
The map doesn’t just tell you where things are — it stays current. And that might be the most important part. Brian told me: “My rule of thumb has always been that roads change at 1 to 2% a year, and that local businesses change at 10 to 20% a year. Keeping your map up to date is a really big challenge. You need people on the ground.”
I spent six years at Google working on what’s called a Visual Positioning System — the tech that uses cameras, LiDAR, and 3D maps to let a device know exactly where it is down to the centimeter. GPS can find your position within five meters of accuracy — and five meters is the difference between a sidewalk and the middle of the road. A delivery robot doesn’t care about the highway. It needs to know: is there a curb here? A bench? A set of stairs?
That’s exactly what Niantic is now building with this data — centimeter-level accuracy, from the ground up, using the exact scans that players collected while catching Pokémon. Traditional maps can’t keep up. A map built by satellites gets updated maybe once a year. A map that gets updated every time someone plays a game? That can keep pace with the real world.
And now they’re taking it further with a neural version of this tech. You can think of it like Rainbolt — the GeoGuessr player who can identify a location from a single blurry photo? Brian put it this way: “I like to think of him as building a neural net in his brain from studying hundreds of thousands or millions of these pictures. And what I think we are talking about with a large geospatial model is to reproduce that neural net by feeding it not a million photos, but hundreds of billions of photos.”
A Large Geospatial Model that can localize you anywhere on Earth from a single photo. That’s where this is headed. And the training data? Already collected. By people catching Pikachu.
Why people are actually upset
And I get it. Because the anger isn’t really about what the data contains. It’s about how the data was collected.
People scanned those Pokéstops to make the game better. They wanted a higher-quality AR experience in their park. They wanted their neighborhood to look good in the game. They were proud of their local landmarks and wanted them represented well. Brian confirmed this:
“People are really proud of their neighborhood, of their city, of their landmarks. And so being able to have a high quality representation of their neighborhood is, I think, a strong motivator.”
That’s a genuine, wholesome motivation. You’re making your game better. You’re contributing to your community’s digital representation. Cool.
And then one day, you find out that the same data is now training a delivery robot to navigate a sidewalk.
That feels different. Even if the terms of service technically allowed it. Even if the data is anonymized. Even if the use case is completely benign. It feels different because the intent shifted underneath you. You were playing a game. You weren’t signing up to be a data collector for a robotics company.
Brian himself acknowledged the side effect nature of it:
“The games themselves were not designed to build this map. The map became a follow-on side effect of making the games better.”
And that’s honest. But “side effect” is a complicated word when twenty million people are the ones producing it. The side effect of what? Of having fun? Of catching Pokémon? That’s not how side effects usually work. Usually, there’s a pill involved, and someone tells you about the side effects before you take it.
The thing is — Niantic isn’t actually the villain here. They split the company. The gaming division was sold to Scopely for $3.5 billion. The spatial platform spun out as Niantic Spatial. Players can opt in to scanning. The scans get anonymized. The incentives are actually more aligned now than they were before.
But the feeling remains. And that feeling — “I was doing something for one reason, and it got used for another” — is worth sitting with. Because this pattern doesn’t stop at Pokémon Go.
Everyone else is mapping the world too
And lest we forget — Niantic is just one of many companies building their own map of the world right now. They’re all capturing different facets of reality:
🚶 Person-level: Axon body cams on hundreds of thousands of officers. Meta Ray-Ban glasses capturing first-person POV at scale — overseas operators reviewing images every time someone says “Hey Meta.” They were in hot water for that recently.
🚗 Vehicle-level: Tesla dashcams on every car in the fleet, massive onboard compute extracting and distilling data to the cloud. Waymo with centimeter-accurate 3D maps of every city they operate in. Fleet telematics cameras on delivery vehicles globally.
🏠 Street & home-level: Flock Safety deploying CCTV across neighborhoods and cities. Amazon with Ring cameras on every doorstep and mailroom — they recently got dragged over that Super Bowl commercial about fusing all their cameras together to find a lost dog. The backlash was so bad they walked the whole thing back. Plus dashcams on every Prime delivery van. Roomba mapping your floor plan every time it vacuums — Amazon wanted that data badly enough to try acquiring iRobot for $1.7 billion before regulators shut it down.
🥽 Headset-level: Apple Vision Pro and Meta Quest build a 3D model of whatever room you’re in every time you put them on. That’s how they know not to let you walk into a wall. Between Ring, Roomba, and your headset — your entire home is being spatially understood by at least three different companies.
📍 Platform-level: Google with Street View cars, aerial planes, satellite imagery, and live location from every Android phone in your pocket. Apple is doing the same with mapping cars — and every LiDAR iPhone is a 3D scanner. And yeah, despite the “Apple is too privacy-conscious” narrative, they’re collecting location data too.
🏃 Trajectory-level: Strava mapped every running and cycling trail on Earth. And accidentally exposed secret military bases in Afghanistan and Syria because soldiers logged their jogs. When you aggregate enough individual trajectories, patterns emerge that were never supposed to be visible. Even Uber — when they plotted millions of rides in Toronto, they essentially reproduced the blueprint of the city. All from location data.
🛰️ Space-level: Planet Labs imaging the entire Earth’s landmass every single day from orbit. Vantor capturing it in higher detail. Iceye doing it in 3D using synthetic aperture radar. If something changes anywhere on the planet — a building goes up, a forest burns down, a military convoy moves — before-and-after imagery within twenty-four hours.
Fused together — body cam, dashcam, doorbell, phone, headset, satellite — every layer of physical reality is being mapped by somebody right now. Different sensors, different angles, different purposes capturing different facets of reality. And most of it is way more invasive than Pokémon Go scanning a statue in a park.
Pokémon Go cracked the incentive problem
The interesting part is how they incentivize the collection. Because mapping the world is expensive.
Google spends massive opex and subsidizes it with local ads and the maps platform. Mapillary tried altruism — asking dashcam owners to contribute footage to departments of transportation. That worked until Meta bought them. Hivemapper pays drivers with crypto.
Pokémon Go cracked something none of them could — a game mechanic that subsidizes the scanning behavior. You’re not building a map. You’re catching Pokémon, or unlocking where one can be caught. The map is just a side effect.
And that game mechanic is honestly kind of brilliant. But it’s also why people feel the way they feel. Because the more fun the incentive, the less it feels like you’re producing valuable data. And the less it feels like you’re producing valuable data, the more jarring it is when someone tells you it’s worth $3.5 billion.
3D scanning is still a niche hobby for reality capture nerds like me. The moment somebody gamifies dense 3D capture at scale — that’s when this blows wide open. And honestly? Looking at the trajectory? That moment might already be here.
What this is really about
The Pokémon Go backlash is real, and the feelings are valid. But it’s also kind of a misdirection. Niantic mapped statues in parks. Meanwhile, Tesla is mapping every road from every angle. Amazon tried to acquire your floor plan. Strava exposed military bases. Planet Labs photographs the entire planet every single day.
If you’re going to be upset about spatial data collection — and there are good reasons to be — Pokémon Go probably shouldn’t be at the top of your list. It should be somewhere in the middle, right between your Roomba and your headset.
Gaming is becoming a way for humans to contribute real-world trajectories that help physical AI learn about the real world. Google does it with live traffic. Tesla does it with autopilot. The mechanic is different but the pattern is identical. The map is always a side effect. The side effect is always more valuable than the original product. And the people who produced the data almost never know what it’s going to be used for until after the fact.
That’s the conversation worth having. Not whether Niantic violated your trust by licensing park scans to a delivery robot company. But whether the “side effect” model of spatial data collection — where the map is a byproduct of something else you wanted to do — is sustainable when the maps become this valuable.
Because the next time someone builds a fun app that asks you to point your camera at stuff, you should probably ask: what’s the side effect?
If you want the full visual breakdown with all the layers mapped out, here’s the video:
If you want the deep dive on how Visual Positioning Systems work and why every major tech company is racing to build one, we made a full video on that too: VPS Video
If this gave you something to think about, share it with fellow reality mappers. The future’s too interesting to navigate alone.
Cheers,
Bilawal Sidhu