LiteLLM Instrumentation - LangWatch

LangWatch integrates with LiteLLM to capture detailed information about your LLM calls across multiple providers through a unified interface.

Installation

pip install langwatch litellm

uv add langwatch litellm

Usage

The LangWatch API key is configured by default via the LANGWATCH_API_KEY environment variable.

Use autotrack_litellm_calls() to automatically capture all LiteLLM calls within a trace.

import langwatch
import litellm
import os
import asyncio
from typing import cast
from litellm import CustomStreamWrapper
from litellm.types.utils import StreamingChoices

langwatch.setup()


@langwatch.trace(name="LiteLLM Autotrack Example")
def get_litellm_response_autotrack(user_message: str):
    langwatch.get_current_trace().autotrack_litellm_calls(litellm)

    response = litellm.completion(
        model="groq/llama-3.1-8b-instant",
        messages=[
        {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": user_message},
        ],
    )

    return response.choices[0].message.content


if __name__ == "__main__":
    reply = get_litellm_response_autotrack("Tell me a joke")
    print("AI Response:", reply)

The @langwatch.trace() decorator creates a parent trace, and autotrack_litellm_calls() enables automatic tracking of all LiteLLM calls for the duration of that trace.

Capturing RAG - Learn how to capture RAG data from retrievers and tools
Capturing Metadata and Attributes - Add custom metadata and attributes to your traces and spans
Capturing Evaluations & Guardrails - Log evaluations and implement guardrails in your LiteLLM applications