Introduction
llmBreakr is a self-hosted AI gateway that puts OpenAI, Anthropic, and Gemini behind one OpenAI-compatible endpoint.
What is llmBreakr?
llmBreakr is a self-hosted AI gateway — a single, OpenAI-compatible API that sits in front of OpenAI, Anthropic, and Gemini. Instead of wiring every service to every provider's SDK, you point your applications at one endpoint, and llmBreakr handles routing, authentication, rate limits, budgets, and logging.
Everything runs inside your own infrastructure. No request or token ever transits a third-party SaaS platform — llmBreakr is the proxy, not a hop before one.
Quickstart
Get a gateway running with Docker Compose in under five minutes.
Architecture
Understand the control plane / data plane split.
API Reference
Call /v1/chat/completions against any configured provider.
Self-hosting
Run it in production with your own database and Redis.
Why a gateway?
Every team that ships more than one LLM feature runs into the same problems:
- Provider sprawl. OpenAI, Anthropic, and Gemini each have their own SDK, auth model, and request shape.
- Raw keys everywhere. Provider API keys end up hardcoded in services, scripts, and CI — with no way to scope, rotate, or revoke them per-project.
- No spend or rate control. A single runaway job can blow through a budget with no circuit breaker in place.
- No audit trail. When something goes wrong, there's no record of which project made which call, with which model, at what cost.
llmBreakr solves this at the infrastructure layer, once, instead of in every service that calls an LLM.
Core capabilities
- Unified endpoint — a single, OpenAI-compatible
/v1/chat/completionsAPI (streaming included) in front of every configured provider. - Virtual API keys — scoped, project-level credentials that replace raw provider keys, with approval workflows before they go live.
- Project-based routing — control exactly which models each project is allowed to call.
- Rate limits & budgets — enforce request-rate ceilings and spend caps per project.
- Full audit trail — every chat request and every admin action is logged.
- RBAC — role-based access control for admin users managing the gateway.
- Management dashboard — a Next.js UI for configuration, keys, and monitoring, bundled in the same deployable.
How it's licensed
llmBreakr is MIT licensed — free to self-host, modify, and use commercially, with the source at github.com/yashb007/llmBreakr-ai-gateway.