Skip to main content
This walkthrough takes you from zero to a traced, budget-gated LLM response.

​
Are you a developer or an admin?

The fastest path depends on which side of the rollout you’re on. Pick one before continuing:

I'm a developer

Your admin already wired up the gateway and you got an SSO sign-in link or asked to install langwatch.Skip ahead to Personal IDE keys: langwatch login --device provisions your virtual key automatically. You don’t need to set up providers, budgets, or routing.

I'm an org admin

Your company just bought governance and you’re rolling it out for the team.Skip ahead to Admin setup: connect SSO, providers, and routing policies once, every developer auto-onboards from there.
The rest of this page covers the service-virtual-key flow used for non-personal traffic (server-to-server calls from your applications, evaluators, prompt playground, etc.). If that’s what you’re here for, read on.
You’ll need a LangWatch account, an organisation admin or project admin role, and at least one model provider (OpenAI, Anthropic, Azure OpenAI, Bedrock, Vertex, or Gemini) configured at Settings → Model Providers. If you haven’t set one up yet, start there, the AI Gateway reuses those credentials.

Python, TypeScript SDK

Drop-in for the OpenAI, Anthropic SDKs, swap base_url and api_key.

Use it from your coding assistant

Wire Claude Code, Codex, OpenCode, Cursor, or Aider through the gateway.

Core concepts

Virtual keys, budgets, ModelProvider gateway settings, routing policies, cache rules, fallback chains.

API reference

OpenAI-compatible /v1/chat/completions + Anthropic-shape /v1/messages.

​
1. Create a virtual key

  1. Open AI Gateway → Virtual Keys in the LangWatch app.
  2. Click New virtual key.
  3. Name it, e.g. my-first-vk.
  4. Select which provider credentials it may use. The first one becomes the primary; later ones become the fallback chain.
  5. (Optional) Attach a budget, pick a cache mode, attach guardrails.
  6. Click Create.
The full secret (vk-lw-01HZX9K3M…) is displayed exactly once. Copy it now and store it in a secret manager. After you dismiss the dialog, LangWatch can never show it again, only the prefix vk-lw-01HZX9 remains visible in the list.

​
2. Call the gateway (OpenAI-compatible)

Using curl: 
curl https://gateway.langwatch.ai/v1/chat/completions \
  -H "Authorization: Bearer $LW_VK" \
  -H "Content-Type: application/json" \
  -d '{
        "model": "gpt-5-mini",
        "messages": [{"role": "user", "content": "Say hi in one word"}]
      }'
The response is OpenAI-shaped. Response headers tell you what happened inside the gateway: 
X-LangWatch-Request-Id: grq_01HZX9K3M…
X-LangWatch-Provider: openai
X-LangWatch-Model: gpt-5-mini
X-LangWatch-Cache: miss
X-LangWatch-Fallback-Count: 0

​
Using the OpenAI SDKs

Python: 
from openai import OpenAI

client = OpenAI(
    base_url="https://gateway.langwatch.ai/v1",
    api_key="vk-lw-...",
)

resp = client.chat.completions.create(
    model="gpt-5-mini",
    messages=[{"role": "user", "content": "Hi"}],
)
print(resp.choices[0].message.content)
TypeScript: 
import OpenAI from "openai";

const openai = new OpenAI({
  baseURL: "https://gateway.langwatch.ai/v1",
  apiKey: process.env.LW_VK,
});

const resp = await openai.chat.completions.create({
  model: "gpt-5-mini",
  messages: [{ role: "user", content: "Hi" }],
});

​
Anthropic-compatible (Claude SDK, Claude Code)

curl https://gateway.langwatch.ai/v1/messages \
  -H "x-api-key: $LW_VK" \
  -H "anthropic-version: 2023-06-01" \
  -H "Content-Type: application/json" \
  -d '{
        "model": "claude-haiku-4-5-20251001",
        "max_tokens": 64,
        "messages": [{"role": "user", "content": "Hi"}]
      }'

​
3. See the trace in LangWatch

Within ~30 seconds the request appears in the project the virtual key belongs to:
  • Messages view shows the prompt, response, tokens, cost, provider, cache outcome, and the full trace.
  • Analytics aggregates across all VKs in the project.
  • Evaluations can run on the traffic passing through the gateway.
Every gateway trace carries these attributes:
AttributeMeaning
langwatch.vk_idVirtual key id.
langwatch.project_idOwning project.
langwatch.team_idOwning team.
langwatch.org_idOwning organisation.
langwatch.principal_idUser or service account that made the call.
gen_ai.usage.cache_read.input_tokens, gen_ai.usage.cache_creation.input_tokensCache economics (OTel GenAI semconv; zero when cache bypassed).
langwatch.fallback.attempt0 on primary success, N for fallback attempts.

​
4. Add a budget

Budgets live one level up from the key, they govern spend across whatever scope you attach them to.
  1. Open AI Gateway → Budgets.
  2. Click New budget.
  3. Pick scope = project, target = the project your VK is in.
  4. Set window = month, limit = $100, on_breach = block.
  5. Save.
Once spend reaches the limit, the next request through any VK in that project returns 402 budget_exceeded. Flip on_breach to warn if you’d rather have a X-LangWatch-Budget-Warning header instead of a hard cut-off. See Budgets for full scoping details.

​
Next steps

  • Concepts: how VKs, budgets, guardrails, and fallback fit together.
  • Fallback chains: survive provider outages with no code change.
  • Caching passthrough: the load-bearing rule for keeping your 90% Anthropic cache discount.
  • CLI Integrations: ship the gateway to every developer’s Claude Code, Codex, Cursor.