I Put Real Planes in the Sky With AR. The 3D Was the Easy Part.

A few years ago I would have told you AR was a graphics problem. You get a 3D model, you get a camera feed, you draw one on top of the other. Ship it.

Then I built location-based AR for real, and the graphics turned out to be the least interesting part of the whole thing.

The build that taught me this was Planes XR, one of the location-based AR experiences I worked on for ARCortex. The pitch is simple to say and brutal to deliver: point your phone at the sky, and the actual aircraft flying overhead show up, positioned where they really are. Not a canned demo of a plane. The plane that is genuinely up there right now, pulled from live flight data, rendered in the correct patch of sky.

If you have only ever done marker-based or surface AR, that one requirement changes almost every engineering decision you make. So I want to walk through why, because the lesson generalizes well past AR.

The three flavors of AR are not the same job