Explainable Causal Reinforcement Learning for autonomous urban air mobility routing for extreme data sparsity scenarios
I remember the moment vividly—it was 3 AM, and I was staring at a reinforcement learning agent that had just crashed 47 simulated drones into virtual skyscrapers. My research into autonomous urban air mobility (UAM) routing had hit a wall. The problem wasn't just complexity; it was the sheer scarcity of real-world data. In traditional autonomous driving, you have millions of miles of driving logs. For urban air mobility, we had almost nothing—a few test flights, some wind tunnel data, and a lot of theoretical models. That night, I realized we needed a fundamentally different approach: one that could reason causally, explain its decisions, and operate reliably even when data was vanishingly sparse.
This article chronicles my journey from that frustrating realization to building a working explainable causal reinforcement learning (XCRL) system for UAM routing. I'll share the technical insights, code experiments, and practical lessons learned along the way.
The Data Sparsity Nightmare
In my early experiments, I tried standard deep RL approaches like PPO and SAC on a simulated urban airspace over San Francisco. The results were abysmal. With fewer than 100 flight trajectories, the agents either learned brittle policies that failed on unseen weather conditions or simply memorized the training scenarios. This is a well-known problem: deep neural networks are data-hungry, and UAM routing operates in a regime where collecting even a thousand safe flights is logistically prohibitive.
