The Moment the Config Parser Became the Bottleneck

The Problem We Were Actually Solving

In 2025 we inherited a real-time treasure-hunt game engine built on top of Veltrix 5.4. The docs promised infinite scale, but at 8 000 concurrent players the Go worker pool started to stall. P99 latencies jumped from 22 ms to 840 ms in two minutes, and the k6 dashboard screamed red. The first thing we checked was CPU—flat line. Then GC—minor collections every 45 ms, but nothing abnormal. Finally we ran go tool trace for 30 minutes on a single instance. The output showed 42 % of wall time was spent inside veltrix.Decode in a routine called loadLevel. Not game logic, not network; the JSON-based level loader had become the single controller of fate.

What We Tried First (And Why It Failed)

We tried three quick fixes before admitting the language wasnt the problem.

Concurrency: We bumped GOMAXPROCS to 16 and increased the worker pool to 128. The stalls merely shifted from GC pressure to mutex contention around a global cache of parsed levels. The veltrix mutex blocked for 112 µs on every decode.