Automatic fallback
Configure an ordered chain of alternate models that a request automatically retries against when the primary provider fails.
When a model's provider fails — an outage, a rate limit, a bad credential — llmBreakr can automatically retry the same request against a different, pre-configured model instead of just returning an error to the caller.
How a request escalates
- The primary model is tried first. A transient failure (any
5xx, or a network error with no status at all) gets one retry on the same model. - If that also fails, or the failure is a rate limit (
429) or a bad credential (401/403), llmBreakr walks the fallback chain in priority order, trying each configured candidate once. - The first candidate that succeeds serves the request. The response includes an
X-LLMBreakr-Served-Modelheader naming it, so you can tell a fallback happened without inspecting logs. - If every candidate is exhausted, the caller sees the primary model's original error — not the last fallback's — since that's what was actually asked for.
A 400 (malformed request) never retries or falls back: it's the same broken request on every provider, so failing fast is correct.
Fallback only applies before any response bytes reach the client. For streaming requests, that means it can only help if the provider connection itself fails to establish — never mid-stream, since silently swapping providers after tokens have already been sent would corrupt the response the client is reading.
Configuring a chain
A fallback chain is scoped to a project, and both the primary and every fallback must already be in that project's model allowlist — a fallback can only ever be a model the project is explicitly granted, never the raw provider catalog directly.
From the dashboard: open a project's detail page → Allowed models → Fallbacks on the model you want to protect → add one or more alternates, reorder them, or remove one.
From the API:
curl -X POST http://localhost:4000/api/admin/project-model-fallbacks \
-H "Authorization: Bearer <access_token>" \
-H "Content-Type: application/json" \
-d '{
"project_id": 4,
"primary_project_model_id": 9,
"fallback_project_model_id": 8,
"priority": 0
}'GET, PATCH (reorder priority), and DELETE on the same route list, reorder, and remove chain entries.
Observability
Every request logged in the audit trail records which model actually served it. When a fallback fires, the log row also carries:
requested_model— the model that was originally asked for (only set when it differs from what served the request)fallback_attempt—0for the primary model,1for the first fallback,2for the second, and so on
Why this isn't the same as load balancing
Fallback is reactive — it only ever activates after a failure. It's deliberately not a way to proactively spread traffic across multiple models on every request; that's a different feature (smart routing) under consideration separately. Fallback chains exist purely for resilience: keep serving the request when the model you actually wanted is unavailable, without the caller having to implement their own cross-provider retry logic.