Skip to main content
If your teams call models through a LiteLLM Proxy, you can send a trace of every request to LangWatch straight from the proxy. Every app, agent, notebook and internal tool behind the proxy shows up in LangWatch, with no SDK in the application code. The proxy exports OpenTelemetry, and LangWatch receives it on its OTLP endpoint. Each request becomes one trace: the HTTP request, authentication, guardrails, the LLM call with model, provider, tokens and cost, and the spend written to the proxy database.
This page is for the LiteLLM Proxy (the gateway server). If you call LiteLLM as a Python library inside your own code, see LiteLLM (SDK) instead.

​
Prerequisites

  • A LiteLLM Proxy you can set environment variables on.
  • A LangWatch API key, from your project settings.

​
Setup

1

Set the environment variables on the proxy

LiteLLM appends /v1/traces to OTEL_ENDPOINT, so the spans land on https://app.langwatch.ai/api/otel/v1/traces.Self-hosted LangWatch: replace https://app.langwatch.ai with the URL of your own instance, for example export OTEL_ENDPOINT="https://langwatch.internal.example.com/api/otel". The proxy only needs network access to your LangWatch instance, so this works for proxies that are only reachable inside your VPC.
2

Start the proxy

No change to your config.yaml is needed:
On Kubernetes, add the same variables to the proxy’s env (or its secret) in your Helm values.
3

Send a request

The trace appears in your LangWatch project within a few seconds, with the model, tokens, cost and latency of the call.

​
Capturing prompts and responses

By default LiteLLM only exports metadata (model, tokens, cost, timing) and never the message content. To see the conversations in LangWatch, and to run evaluations on them, set:
Once the content reaches LangWatch, your Data Privacy rules apply: PII and secrets are redacted before storage, and you can drop or restrict input and output per organization, team or project. The default Essential PII redaction does not catch everything (names and locations, for example, need the Strict level), and captured content is readable by everyone with access to the project. Check your privacy settings before turning capture on. A common setup is to capture at the proxy, restrict input and output in LangWatch, and decide there who can read what.

​
Linking proxy calls to your application traces

LiteLLM continues any trace it receives in the W3C traceparent header. If your application is instrumented with the LangWatch SDK or any OpenTelemetry SDK that propagates context on outgoing HTTP calls, the proxy’s spans nest inside your application’s trace, so you see your agent’s steps and the gateway call in one tree.

​
Sending to LangWatch and another backend

LiteLLM can export the same traces to more than one destination, for example LangWatch plus an existing Datadog or Grafana setup. See LiteLLM’s OpenTelemetry v2 docs for how to combine destinations.

​
Older LiteLLM versions

If your proxy does not support LITELLM_OTEL_V2 yet, use the original OpenTelemetry callback. It takes the full traces URL:
config.yaml
Upgrading to a version with OpenTelemetry v2 is recommended: it produces one clean trace per request and follows the OpenTelemetry GenAI semantic conventions.

​
Troubleshooting

  • No traces: check the proxy logs for exporter errors. A 401 means the API key in OTEL_HEADERS is wrong or belongs to a different LangWatch instance.
  • Traces without messages: set OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT, see above.
  • Self-hosted: make sure OTEL_ENDPOINT ends in /api/otel (v2) or /api/otel/v1/traces (the older callback).
Also check: OpenTelemetry integration guide for the endpoint in any language, and Data Privacy for redaction and access rules.
Last modified on September 25, 2026