Python

What changes in your code

Two settings change: the base URL points at the gateway, and the API key is a virtual key. The request body, streaming, tools and every other SDK feature stay the same. Name the model as provider/model on the OpenAI-shaped and Anthropic-shaped routes; see Model naming.

SDK	Base URL	Where the key goes
OpenAI	`https://gateway.langwatch.ai/v1`	`api_key`, sent as `Authorization: Bearer`
Anthropic	`https://gateway.langwatch.ai`	`api_key`, sent as `X-Api-Key`
Google GenAI	`https://gateway.langwatch.ai`	`api_key`, sent as `X-Goog-Api-Key`
ElevenLabs	`https://gateway.langwatch.ai`	`api_key`, sent as `Xi-Api-Key`

OpenAI SDK

from openai import OpenAI

client = OpenAI(
    base_url="https://gateway.langwatch.ai/v1",
    api_key="vk-lw-...",
)

response = client.chat.completions.create(
    model="openai/gpt-5-mini",
    messages=[{"role": "user", "content": "Say hi in one word"}],
)
print(response.choices[0].message.content)

The same client serves client.responses, client.embeddings, client.audio.speech and client.audio.transcriptions. With an Anthropic or Gemini model name the gateway translates the chat request to the provider’s format.

Anthropic SDK

import anthropic

client = anthropic.Anthropic(
    base_url="https://gateway.langwatch.ai",
    api_key="vk-lw-...",
)

message = client.messages.create(
    model="anthropic/claude-haiku-4-5",
    max_tokens=64,
    messages=[{"role": "user", "content": "Say hi in one word"}],
)
print(message.content[0].text)

Google GenAI SDK

from google import genai

client = genai.Client(
    api_key="vk-lw-...",
    http_options={"base_url": "https://gateway.langwatch.ai"},
)
response = client.models.generate_content(
    model="gemini-2.5-flash",
    contents="Say hi in one word",
)
print(response.text)

The Google GenAI SDK calls the gateway’s Gemini passthrough at /v1beta, which names the model in Google’s own form. Write gemini-2.5-flash, not gemini/gemini-2.5-flash. The provider/model form applies to the OpenAI-shaped and Anthropic-shaped routes.

LangChain

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    base_url="https://gateway.langwatch.ai/v1",
    api_key="vk-lw-...",
    model="openai/gpt-5-mini",
)

ChatAnthropic takes base_url="https://gateway.langwatch.ai" and the virtual key as api_key.

LlamaIndex

from llama_index.llms.openai import OpenAI

llm = OpenAI(
    api_base="https://gateway.langwatch.ai/v1",
    api_key="vk-lw-...",
    model="openai/gpt-5-mini",
)

Link the gateway span to your trace

The gateway records one gen_ai.<request type> span per request in the trace project of the virtual key. Send a W3C traceparent header and the gateway nests that span under your span. Without it, each request is its own trace. Any OpenTelemetry-instrumented application already has the current context. Inject it into the request headers:

from openai import OpenAI
from opentelemetry.propagate import inject

def trace_headers() -> dict[str, str]:
    carrier: dict[str, str] = {}
    inject(carrier)  # {"traceparent": "00-<trace id>-<span id>-01"} inside an active span
    return carrier

client = OpenAI(base_url="https://gateway.langwatch.ai/v1", api_key="vk-lw-...")

response = client.chat.completions.create(
    model="openai/gpt-5-mini",
    messages=[{"role": "user", "content": "Say hi in one word"}],
    extra_headers=trace_headers(),
)

Call trace_headers() per request, inside the span you want the gateway span under. default_headers on the client is read once and would pin the first span. The LangWatch Python SDK uses OpenTelemetry, so the same code works inside langwatch.trace() or a @langwatch.span() function. Do not also autotrack the OpenAI client for calls that go through the gateway: the gateway already records the span with the model, the tokens and the cost.

Headers you can send

Header	Purpose
`traceparent`	Nests the gateway span in your trace.
`X-LangWatch-End-User-Id`	The end user of your product, for per-end-user budgets and spend. The OpenAI `user` body field works too.
`X-LangWatch-Metadata`	A JSON object, at most 4096 bytes, copied onto the spend event.

Pass them per request with extra_headers={...}. The gateway removes them before it forwards the request to the provider. See Billing and spend events.

Read the response headers

raw = client.chat.completions.with_raw_response.create(
    model="openai/gpt-5-mini",
    messages=[{"role": "user", "content": "Say hi in one word"}],
)
headers = raw.http_response.headers
print(headers["X-LangWatch-Gateway-Request-Id"])  # quote this to support
print(headers.get("X-LangWatch-Budget-Warning"))   # "project:84" when a budget is at 80 percent or more
print(headers.get("Traceparent"))                  # the gateway span in your project
response = raw.parse()

Manage keys and budgets from Python

The LangWatch Python SDK wraps the management routes: langwatch.virtual_keys, langwatch.gateway_budgets, langwatch.spend_events and langwatch.webhooks. They take a LangWatch API key through langwatch.setup(api_key=...). See Virtual keys and Billing and spend events.

Self-hosted gateway

Replace the host with your gateway’s URL. Everything else is the same.

client = OpenAI(base_url="https://gateway.acme.internal/v1", api_key="vk-lw-...")

Troubleshooting

Symptom	Cause
`401` on every request	The key is wrong, revoked, or sent in a header the gateway does not read. The gateway reads `Authorization: Bearer`, `X-Api-Key`, `X-Goog-Api-Key` and `Xi-Api-Key`.
`400 model_not_allowed`	The key restricts its models under Provider access, or its routing policy does.
Two spans per call in Trace Explorer	The OpenAI client is also autotracked by the LangWatch SDK. Remove the autotrack for gateway calls.
The gateway span is its own trace	No `traceparent` on the request, or the headers were computed outside the active span.

See Errors for every error code.

Get Started

SDK Integration

Virtual Keys & Budgets

Billing & Metering

Governance

Providers

Features

API Reference

Self-Hosting

Cookbooks

What changes in your code

OpenAI SDK

Anthropic SDK

Google GenAI SDK

LangChain

LlamaIndex

Link the gateway span to your trace

Headers you can send

Read the response headers

Manage keys and budgets from Python

Self-hosted gateway

Troubleshooting

​What changes in your code

​OpenAI SDK

​Anthropic SDK

​Google GenAI SDK

​LangChain

​LlamaIndex

​Link the gateway span to your trace

​Headers you can send

​Read the response headers

​Manage keys and budgets from Python

​Self-hosted gateway

​Troubleshooting

What changes in your code

OpenAI SDK

Anthropic SDK

Google GenAI SDK

LangChain

LlamaIndex

Link the gateway span to your trace

Headers you can send

Read the response headers

Manage keys and budgets from Python

Self-hosted gateway

Troubleshooting