Stop Using Round-Robin: High-Throughput Java Virtual Thread Routing with P2C
Virtual threads allow Java microservices to comfortably run 50,000 concurrent requests per instance, but your legacy round-robin load balancer is utterly destroying your p99 latency. Blindly distributing traffic without accounting for dynamic queue depth creates catastrophic head-of-line blocking across Loom-enabled nodes.
Why Most Developers Get This Wrong
Assuming uniform execution speed: Round-robin assumes every request takes equal time, which breaks down the moment heavy virtual thread workloads hit asymmetric I/O bottlenecks.
Using raw CPU/Memory metrics for routing: Host-level metrics update every few seconds, making them completely blind to microsecond-level virtual thread queue spikes.
