Current MoE serving pipelines treat every decode worker as interchangeable, assuming that equalizing request counts suffices to minimize latency. In practice, the weight‑loading cost of each request varies wildly because different inputs activate disjoint expert subsets, and ignoring this fact inflates inference time.

Prefill‑decode (PD) disaggregated serving has historically relied on routers that balance only the number of pending tokens per worker, without considering which experts will be needed later. This load‑centric view leaves a hidden dimension—expert locality—that standard schedulers never exploit.

ELDR cuts median time‑per‑output token by 5.9–13.9 % across three MoE models and two workloads while leaving generated images unchanged. “Implemented in vLLM and evaluated on deployments of up to 40 GPUs, ELDR reduces median TPOT by 5.9–13.9% over the strongest of four load‑balancing baselines across three MoE models and two workloads, with model outputs unchanged.” [1]

The router’s premise holds because prefill‑time expert activations predict decode‑time usage. “The correlation is strong for Qwen3-30B-A3B and Gemma-4‑26B‑A4B, and substantial for GPT‑OSS‑120B.” [1] This predictive signal lets the system assign each request to a worker whose cached experts already match its signature.