The Counterintuitive Truth: Feature Flags Are Not Config Files
Most engineers first encounter feature flags as a simple abstraction: a key-value lookup that returns true or false. That mental model works fine for a single service handling a few hundred requests per minute. It becomes actively dangerous at scale.
A mature feature flag system isn't a config file with an API wrapper — it's a distributed control plane. The distinction matters architecturally. A control plane manages the real-time behavior of a running system across many nodes simultaneously, with its own consistency guarantees, failure semantics, and propagation latency. That's a fundamentally different design problem than reading a YAML file on startup.
One constraint drives every downstream decision: user traffic must never block on a remote flag service call. If evaluation requires a synchronous RPC, you've coupled your request path to the availability and latency of an external system. Netflix's Archaius library enforces this by evaluating flags entirely in-process against a locally-cached configuration snapshot. A network round-trip per evaluation injects 10–50ms of tail latency at p99 — catastrophic when you're competing on streaming start times measured in hundreds of milliseconds. Google, Meta, and Netflix collectively evaluate flags against millions of requests per second with sub-millisecond overhead. That figure is only achievable through local evaluation backed by an async synchronization layer, not RPC.
