Fallback rules are where an AI API gateway becomes operationally valuable.

The goal is not to blindly retry every failed LLM call. The goal is to choose the right backup model, provider, or budget path based on the workflow, customer tier, latency target, and risk of a lower-quality answer.

A practical fallback policy should define:

which failures are retryable;

which workflows may downgrade models;