Google Vertex AI Instrumentation

LangWatch offers robust integration with Google Vertex AI, allowing you to capture detailed information about your Vertex AI API calls automatically. The recommended approach is to use OpenInference instrumentation, which provides comprehensive tracing for Google Vertex AI API calls and integrates seamlessly with LangWatch.

Using OpenInference Instrumentation

The recommended approach for instrumenting Google Vertex AI calls with LangWatch is to use the OpenInference instrumentation library, which provides comprehensive tracing for Google Vertex AI API calls.

What OpenInference Captures

The OpenInference Vertex AI instrumentation automatically captures:

LLM Calls: All text generation, chat completion, and embedding requests
Model Information: Model name, version, and configuration parameters
Input/Output: Prompts, responses, and token usage
Performance Metrics: Latency, token counts, and cost information
Error Handling: Failed requests and error details
Context Information: Session IDs, user IDs, and custom metadata

Installation and Setup

Prerequisites

Install the OpenInference Vertex AI instrumentor:

pip install openinference-instrumentation-vertexai

Install LangWatch SDK:
pip install langwatch

Set up your Google Cloud credentials:

export GOOGLE_APPLICATION_CREDENTIALS="path/to/your/service-account-key.json"
export GOOGLE_CLOUD_PROJECT="your-project-id"
export GOOGLE_CLOUD_LOCATION="us-central1"

The LangWatch API key is configured by default via the LANGWATCH_API_KEY environment variable.

Basic Setup

There are two main ways to integrate OpenInference Vertex AI instrumentation with LangWatch:

1. Via `langwatch.setup()` (Recommended)

You can pass an instance of the VertexAIInstrumentor to the instrumentors list in the langwatch.setup() call. LangWatch will then manage the lifecycle of this instrumentor.

import langwatch
from vertexai.language_models import TextGenerationModel
import os

# Example using OpenInference's VertexAIInstrumentor
from openinference.instrumentation.vertexai import VertexAIInstrumentor

# Initialize LangWatch with the VertexAIInstrumentor
langwatch.setup(
    instrumentors=[VertexAIInstrumentor()]
)

# Initialize Vertex AI
from vertexai import init
init(project=os.getenv("GOOGLE_CLOUD_PROJECT"), location=os.getenv("GOOGLE_CLOUD_LOCATION"))

model = TextGenerationModel.from_pretrained("text-bison@001")

@langwatch.trace(name="Vertex AI Call with OpenInference")
def generate_text_with_openinference(prompt: str):
    # No need to call autotrack explicitly, the OpenInference instrumentor handles Vertex AI calls globally.
    response = model.predict(prompt)
    return response.text

if __name__ == "__main__":
    user_query = "Tell me a joke about Python programming."
    response = generate_text_with_openinference(user_query)
    print(f"User: {user_query}")
    print(f"AI: {response}")

2. Direct Instrumentation

If you have an existing OpenTelemetry TracerProvider configured in your application, you can use the instrumentor’s instrument() method directly. LangWatch will automatically pick up the spans generated by these instrumentors as long as its exporter is part of the active TracerProvider.

import langwatch
from vertexai.language_models import TextGenerationModel
import os
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import SimpleSpanProcessor, ConsoleSpanExporter

from openinference.instrumentation.vertexai import VertexAIInstrumentor

langwatch.setup()

# Initialize Vertex AI
from vertexai import init
init(project=os.getenv("GOOGLE_CLOUD_PROJECT"), location=os.getenv("GOOGLE_CLOUD_LOCATION"))

model = TextGenerationModel.from_pretrained("text-bison@001")

# Instrument Vertex AI directly using the OpenInference library
VertexAIInstrumentor().instrument()

@langwatch.trace(name="Vertex AI Call with Direct OpenInference Instrumentation")
def get_story_ending(beginning: str):
    response = model.predict(
        f"You are a creative writer. Complete the story: {beginning}"
    )
    return response.text

if __name__ == "__main__":
    story_start = "In a land of dragons and wizards, a young apprentice found a mysterious map..."
    ending = get_story_ending(story_start)
    print(f"Story Start: {story_start}")
    print(f"AI's Ending: {ending}")

Which Approach to Choose?

OpenInference Instrumentation is recommended for most use cases as it provides comprehensive, automatic instrumentation with minimal setup
Direct OpenTelemetry Setup is useful when you need fine-grained control over the tracing configuration or are already using OpenTelemetry extensively

Both approaches effectively send Vertex AI call data to LangWatch for monitoring and analysis.

Overview

SDKs

Frameworks

Model Providers

Tools

No-Code Platforms

Direct Integrations

Google Vertex AI Instrumentation

Using OpenInference Instrumentation

What OpenInference Captures

Installation and Setup

Prerequisites

Basic Setup

1. Via `langwatch.setup()` (Recommended)

2. Direct Instrumentation

Which Approach to Choose?

​Using OpenInference Instrumentation

​What OpenInference Captures

​Installation and Setup

​Prerequisites

​Basic Setup

​1. Via langwatch.setup() (Recommended)

​2. Direct Instrumentation

​Which Approach to Choose?

Using OpenInference Instrumentation

What OpenInference Captures

Installation and Setup

Prerequisites

Basic Setup

1. Via `langwatch.setup()` (Recommended)

2. Direct Instrumentation

Which Approach to Choose?