I have a small cooperative multi-agent reinforcement-learning setup: eight agents on a 100×100 grid, learning to reach three goals while gathering resources and avoiding obstacles. Standard MAPPO — a shared actor, a centralized critic, per-agent advantages, the usual PPO machinery. Each agent sees the world through an 11×11 egocentric window.
That window is the whole story, and it took an ablation to make me take it seriously.
The setup, and the thing that's easy to miss
Look at the numbers for a second. The grid is 100×100. The agent sees an 11×11 patch centered on itself — reach ±5 in each direction. Goals are placed uniformly at random. So on any given step, what's the chance a goal is actually inside the patch an agent can see?
Almost never. A goal is a small target on a 10,000-cell map, and each agent perceives roughly 1% of that map at a time. For the overwhelming majority of steps, the agent's observation contains no signal about where it's supposed to go. It's a partially observed task in the most literal sense: the objective is usually outside the sensory window entirely.
