Concepts
Rate limits & budgets
Per-project request-rate ceilings and spend enforcement, backed by Redis.
Every project can carry two independent limits, enforced before a request reaches a provider:
- Rate limits — a ceiling on requests over a time window, protecting against runaway loops and misbehaving clients.
- Budgets — a spend cap, tracked from provider usage/cost data, that stops a project once it's exhausted.
Why enforce before the provider call
Because limit checks happen in the data-plane pipeline (see Architecture) before the provider adapter runs, an over-budget or rate-limited request never reaches OpenAI, Anthropic, or Gemini — you don't pay for calls that get rejected.
Backed by Redis
Rate and spend counters live in Redis, so enforcement stays fast under concurrent load without hitting MySQL on every request. Redis is one of only two external dependencies llmBreakr needs (the other is MySQL) — see Installation.
Where this shows up
- A project past its request-rate ceiling gets an immediate rejection, not a slow queue.
- A project past its budget is blocked from further calls until the budget is raised or reset by an admin.
- Both are configured per project from the dashboard, alongside model access.