Skip to main content
To cap LLM spend per team, you put an AI gateway such as the LangWatch AI Gateway between your code and the providers and give every app, team or person its own virtual key instead of a provider key. Each key carries a budget in USD, and the gateway blocks requests once the budget is spent. It does this for OpenAI, Anthropic, Azure OpenAI, AWS Bedrock, Google Vertex AI, Gemini and the other providers. Your code keeps the OpenAI or Anthropic SDK and changes two values, the base URL and the API key. To start now, go to Create a virtual key with a budget.

​
Why a virtual key per app

With one shared provider key, the provider bill tells you only the total. You can’t say which team spent it, and you can’t stop one runaway agent without cutting off everyone using that key. A virtual key stands in for the provider key. The provider keys stay in LangWatch, and each key you hand out can only reach the providers you allow. You can rotate, disable or revoke it without touching the provider account.

​
1. Create a virtual key with a budget

Add your provider keys once, under Settings > Model Providers, then create the key:
  1. Open AI Gateway > Virtual Keys and click New virtual key.
  2. Name it after the app or team, for example support-agent.
  3. Keep Ownership on Project, so the key’s traces and costs land in that project, or set it to the team.
  4. Under Budget, enter a monthly limit, for example $50, and the key is blocked when it reaches it.
  5. Under Provider access, keep All providers or pick the ones this key may call.
  6. Click Create and copy the secret. It is shown once.
From the CLI (see Manage keys from the CLI), the same key on a team with a $100 monthly budget:

​
2. Point your code at the gateway

Your code changes only two values: the API key becomes the virtual key, and the base URL becomes the gateway. The request body stays the same:
The Anthropic SDK works the same way with base_url="https://gateway.langwatch.ai". The gateway also translates between formats, so an OpenAI SDK call with model="anthropic/claude-sonnet-4-5" reaches Anthropic. See Python, TypeScript and Go.

​
3. Add budgets on teams, projects and people

A key’s own budget is one layer. Under AI Gateway > Budgets, click New budget to cap a whole Team, Project or Organization, or each Member, as well. A Group budget is not a shared cap: it gives every member of the group their own allowance. For one cap the whole team shares, use a Team budget. Every request is checked against every budget that applies to it, so a request on support-agent counts against the key, its project, its team and the organization at once, and the first block budget at its limit stops it. A warn budget at its limit still serves the request, and a block budget filtered to one provider only takes that provider out of the request’s chain. Each budget has a window (per day, week, calendar month, or no reset) and an action at the limit: The 402 is an OpenAI-shaped error whose error.meta.budget_scope says which budget ran out, so your app can tell its users something useful. Webhooks fire at 80 percent and at the limit, on warn budgets too. If you build a product on top of LLMs, an end-user budget gives each of your own customers a cap, read from the X-LangWatch-End-User-Id header or the OpenAI user field.

​
4. See who spends what

Open AI Gateway > Usage for total spend, cost per request, and the top keys and models over a period. The Virtual Keys list shows each key’s spend this month. Every request is also a trace in Trace Explorer with the model, the tokens, the cost and the virtual key that sent it. So when a team’s spend jumps, you can open the requests behind it and see which prompts and which model caused it, in the same place you debug the app.

​
Limits to know

Spend can pass a budget before the gateway stops it, so set the limit below what you can afford to lose:
  • A budget can run slightly past its limit. The gateway checks before a request and counts the cost after it, so a request admitted under the limit is served in full whatever it ends up costing, requests in flight at the same time all pass before their costs are counted, and a key can keep serving for up to a minute after it crosses the limit.
  • Rate limits count per gateway replica. A key’s requests per minute and per day are enforced on each replica, so with three replicas a key can send up to three times its limit. Tokens per minute are not enforced. See Rate limits.
  • Fallbacks run in order. A routing policy tries providers in the order you list them; it does not balance load across them.

​
Questions about the LangWatch AI Gateway

​
Does the LangWatch AI Gateway work as a LiteLLM alternative

Yes, it works as a LiteLLM alternative. Keys, budgets, team budgets, model lists, end-user ids and fallbacks all have an equivalent, and LiteLLM’s x-litellm-end-user-id header is accepted as is. Migrate from LiteLLM maps each setting and moves an app over with a base URL change.

​
Can I self-host the LangWatch AI Gateway

Yes, it ships in the LangWatch Helm chart next to the app. See Helm chart.

​
How does the LangWatch AI Gateway calculate spend

From the token counts the provider reports and the LangWatch pricing catalog. Requests the gateway rejected, and requests that failed at the provider, are not charged. Next, follow the Quickstart to send a first request and see its trace and its spend, or read Budgets for every scope and window.
Last modified on November 10, 2026