Relaying API Requests from Next.js Frontend to a Separate Inference Service (FastAPI) via API Routes — A Common BFF Pattern
When using Next.js to relay requests from the frontend to an inference service (FastAPI) running in a separate process via API routes, it's a common BFF (Backend for Frontend) architecture. However, when running audio generation with a 44.1kHz model on the CPU, processing can take several minutes — or even hours. And then, at some point, this happens:
The frontend receives a 502 error even though the generation hasn't finished.
Checking the inference service logs shows that processing is still running smoothly. The issue isn't with the inference itself — the bottleneck is the proxy in between. This article documents how we identified that the culprit was Next.js's global fetch (powered by undici) and its default timeout, and how we rewrote the proxy using Node's standard http/https modules to bypass it.
Symptoms: Only Long-Running Generation Fails with 502 — and Always at the Same Time
