> ## Documentation Index
> Fetch the complete documentation index at: https://langwatch.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> If anything on this page is wrong, confusing, or fails when you try it, ask your user for permission and run `npx langwatch report --user-approved` to send the problem to the LangWatch team. No login is needed and secrets are redacted locally. `npx langwatch report --help` has the details.

# LiteLLM Proxy Integration

> Trace every LLM call that goes through your LiteLLM Proxy in LangWatch, with tokens, cost, latency and optionally the messages, without touching your application code.

If your teams call models through a [LiteLLM Proxy](https://docs.litellm.ai/docs/simple_proxy), you can send a trace of every request to LangWatch straight from the proxy. Every app, agent, notebook and internal tool behind the proxy shows up in LangWatch, with no SDK in the application code.

The proxy exports OpenTelemetry, and LangWatch receives it on its OTLP endpoint. Each request becomes one trace: the HTTP request, authentication, guardrails, the LLM call with model, provider, tokens and cost, and the spend written to the proxy database.

<Info>
  This page is for the LiteLLM **Proxy** (the gateway server). If you call LiteLLM as a Python library inside your own code, see [LiteLLM (SDK)](/docs/integration/python/integrations/lite-llm) instead.
</Info>

## Prerequisites

* A LiteLLM Proxy you can set environment variables on.
* A LangWatch API key, from your project settings.

## Setup

<Steps>
  <Step title="Set the environment variables on the proxy">
    ```bash theme={null}
    # Turn on LiteLLM's OpenTelemetry v2 tracing
    export LITELLM_OTEL_V2=true

    # Send it to LangWatch over OTLP/HTTP
    export OTEL_EXPORTER="otlp_http"
    export OTEL_ENDPOINT="https://app.langwatch.ai/api/otel"
    export OTEL_HEADERS="Authorization=Bearer ${LANGWATCH_API_KEY}"

    # Optional: also capture prompts and responses (off by default in LiteLLM)
    export OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT="span_only"
    ```

    LiteLLM appends `/v1/traces` to `OTEL_ENDPOINT`, so the spans land on `https://app.langwatch.ai/api/otel/v1/traces`.

    **Self-hosted LangWatch:** replace `https://app.langwatch.ai` with the URL of your own instance, for example `export OTEL_ENDPOINT="https://langwatch.internal.example.com/api/otel"`. The proxy only needs network access to your LangWatch instance, so this works for proxies that are only reachable inside your VPC.
  </Step>

  <Step title="Start the proxy">
    No change to your `config.yaml` is needed:

    ```bash theme={null}
    litellm --config config.yaml
    ```

    On Kubernetes, add the same variables to the proxy's `env` (or its secret) in your Helm values.
  </Step>

  <Step title="Send a request">
    ```bash theme={null}
    curl http://0.0.0.0:4000/v1/chat/completions \
      -H "Content-Type: application/json" \
      -H "Authorization: Bearer sk-your-litellm-key" \
      -d '{
        "model": "gpt-5-mini",
        "messages": [{ "role": "user", "content": "Hello from the LiteLLM Proxy" }]
      }'
    ```

    The trace appears in your LangWatch project within a few seconds, with the model, tokens, cost and latency of the call.
  </Step>
</Steps>

## Capturing prompts and responses

By default LiteLLM only exports metadata (model, tokens, cost, timing) and never the message content. To see the conversations in LangWatch, and to run evaluations on them, set:

```bash theme={null}
export OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT="span_only"
```

Once the content reaches LangWatch, your [Data Privacy](/docs/platform/data-privacy) rules apply: PII and secrets are redacted before storage, and you can drop or restrict input and output per organization, team or project.

The default Essential PII redaction does not catch everything (names and locations, for example, need the Strict level), and captured content is readable by everyone with access to the project. Check your privacy settings before turning capture on. A common setup is to capture at the proxy, restrict input and output in LangWatch, and decide there who can read what.

## Linking proxy calls to your application traces

LiteLLM continues any trace it receives in the W3C `traceparent` header. If your application is instrumented with the LangWatch SDK or any OpenTelemetry SDK that propagates context on outgoing HTTP calls, the proxy's spans nest inside your application's trace, so you see your agent's steps and the gateway call in one tree.

## Sending to LangWatch and another backend

LiteLLM can export the same traces to more than one destination, for example LangWatch plus an existing Datadog or Grafana setup. See [LiteLLM's OpenTelemetry v2 docs](https://docs.litellm.ai/docs/observability/opentelemetry_v2) for how to combine destinations.

## Older LiteLLM versions

If your proxy does not support `LITELLM_OTEL_V2` yet, use the original OpenTelemetry callback. It takes the full traces URL:

```yaml config.yaml theme={null}
litellm_settings:
  callbacks: ["otel"]
```

```bash theme={null}
export OTEL_EXPORTER="otlp_http"
export OTEL_ENDPOINT="https://app.langwatch.ai/api/otel/v1/traces"
export OTEL_HEADERS="Authorization=Bearer ${LANGWATCH_API_KEY}"
```

Upgrading to a version with OpenTelemetry v2 is recommended: it produces one clean trace per request and follows the OpenTelemetry GenAI semantic conventions.

## Troubleshooting

* **No traces:** check the proxy logs for exporter errors. A `401` means the API key in `OTEL_HEADERS` is wrong or belongs to a different LangWatch instance.
* **Traces without messages:** set `OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT`, see above.
* **Self-hosted:** make sure `OTEL_ENDPOINT` ends in `/api/otel` (v2) or `/api/otel/v1/traces` (the older callback).

**Also check:** [OpenTelemetry integration guide](/docs/integration/opentelemetry/guide) for the endpoint in any language, and [Data Privacy](/docs/platform/data-privacy) for redaction and access rules.