This is a follow-up to SynaptoRoute v0.3.0: Matching Semantic Router While Scaling to 50,000 Routes. If you're new here: SynaptoRoute is a high-performance semantic routing engine that classifies user queries into deterministic software logic locally, without API calls.
The Wall We Hit
In v0.3.0, we proved that SynaptoRoute could match the accuracy of industry standards on standard benchmarks (Banking77, CLINC150) while retaining <50ms P99 latency across 50,000 dense routes.
But scale isn't just about total capacity. It's about concurrent mutation.
Under heavy asynchronous load, specifically, when a system is attempting to route incoming queries while simultaneously adding hundreds of new routes, the architecture began to show stress fractures. The FaissIndex required global locks to rebuild. FastEmbed mathematical execution was starving the asyncio event loop. SQLite connections threw ProgrammingError exceptions across multiple threads. And our new RedisSyncManager created an O(N^2) broadcast storm when 10 replicas all synced identical state changes simultaneously.
