What the gateway does for you
Your provider API keys stay in LangWatch. You add OpenAI, Anthropic, Azure OpenAI, AWS Bedrock, Google Vertex AI, Google Gemini and the other providers once, under Settings > Model Providers. Every application, team or person gets a virtual key instead of a provider key. A virtual key is like a virtual credit card that you can attach a budget to: it is scoped to a project, a team, an organization or one person, it reaches only the providers you allow, and you can rotate, disable or revoke it without touching the provider account. Your code points the OpenAI SDK or the Anthropic SDK athttps://gateway.langwatch.ai and sends the virtual key as the API key. The request body stays the same.
The gateway then gives you, on every request it authenticates:
- Spend per key, person, team and project, in the Usage page and as a spend event you can pull or receive by webhook.
- Budgets on any of those scopes, and per end user of your product, that block the request or add a warning header.
- A trace in Trace Explorer with the model, the tokens, the cost, the virtual key and the end user.
- Rate limits, routing and guardrails on the key, and cache rules on the organization, with no change in application code.
AI Gateway > Virtual Keys: one key per application, its scope, its status and its spend this month.
Endpoints the gateway serves
The gateway translates between wire formats where the provider differs. For example, an OpenAI SDK call to
/v1/chat/completions with model: "anthropic/claude-sonnet-4-5" reaches Anthropic’s Messages API and comes back as an OpenAI chat completion.
Get started
Quickstart
Create a virtual key, send one request, open the trace, see the spend.
Concepts
Virtual key, scope, budget, provider credential, routing policy, spend event.
Python SDK
OpenAI and Anthropic Python SDKs, LangChain, LlamaIndex, trace propagation.
TypeScript SDK
OpenAI and Anthropic TypeScript SDKs, Vercel AI SDK, LangChain.js.
Control spend
Virtual keys
Scopes, provider access, expiry, rotation, disable and revoke, CLI and REST.
Budgets
Caps per organization, team, project, key, person, group or end user. Block or warn.
Rate limits
Requests per minute and per day on a key.
Billing and spend events
One spend event per authenticated request, by webhook or by REST, for rebilling your customers.
Connect providers and tools
Providers
OpenAI, Anthropic, Azure OpenAI, AWS Bedrock, Vertex AI, Gemini, xAI, Groq, Cerebras, DeepSeek, Voyage AI, ElevenLabs, custom endpoints.
Coding Agents
Claude Code, Codex, Gemini CLI, OpenCode, Cursor and Aider through the gateway.
Guardrails
Run LangWatch evaluators on the request or the response.
Self-hosting
Run the gateway next to your LangWatch installation with the Helm chart.