Observability

Where each signal lands

Every request through /v1 produces three things you can read. The one exception is on the trace span: a successful chat or messages call that returns no output, no cost and no completion tokens is dropped before export.

Signal	Where you read it
Trace span	Trace Explorer, in the project the virtual key writes its traces to
Response headers	On the HTTP response your client already receives
Prometheus metrics	`GET /metrics` on the gateway port (`5563` by default), the same listener that serves `/v1/*`

The gateway starts one span per dispatched request and exports it to the project the virtual key names. The span is named gen_ai.<operation>, for example gen_ai.chat.

Trace attributes

The table lists every attribute the gateway sets. An attribute whose value the gateway does not have is left off the span.

Attribute	Meaning
`gen_ai.operation.name`	Request type: `chat`, `messages`, `responses`, `embeddings`, `speech`, `transcription`, `passthrough` or `realtime_session`
`gen_ai.provider.name`	Provider the request was dispatched to
`gen_ai.request.model`	Model dispatched, in the provider-prefixed spelling (`anthropic/claude-haiku-4-5`)
`langwatch.requested_model`	Model name the client sent. Set only when a routing policy rewrote it
`gen_ai.input.messages`	Request messages
`gen_ai.output.messages`	Response messages
`gen_ai.usage.input_tokens`	Input tokens the provider billed fresh, with cache reads and cache writes excluded
`gen_ai.usage.output_tokens`	Completion tokens
`gen_ai.usage.total_tokens`	Total the provider reported
`gen_ai.usage.cache_read.input_tokens`	Input tokens served from the provider’s prompt cache
`gen_ai.usage.cache_creation.input_tokens`	Input tokens written to the provider’s prompt cache
`gen_ai.usage.cache_creation_1h.input_tokens`	The part of the cache writes that bought a one-hour entry, when the provider reports it
`gen_ai.usage.input_audio_tokens`	Audio input tokens, counted apart from `gen_ai.usage.input_tokens`
`gen_ai.usage.output_audio_tokens`	Audio output tokens, counted apart from `gen_ai.usage.output_tokens`
`gen_ai.usage.input_chars`	Characters a text-to-speech call synthesized
`gen_ai.usage.audio_seconds`	Seconds of audio a speech-to-text call transcribed
`gen_ai.usage.cost`	Cost the gateway computed for the call
`gen_ai.conversation.id`	Session id the client sent, so multi-turn calls group into one thread
`langwatch.project_id`	Project the span is filed under
`langwatch.virtual_key_id`	Virtual key that authenticated the request
`langwatch.gateway_request_id`	Same value as the `X-LangWatch-Gateway-Request-Id` response header
`langwatch.model_provider_id`	Model provider row the request was dispatched to. Provider-filtered budgets match on this id
`langwatch.labels`	The virtual key’s tags. Trace Explorer filters on them as Label
`langwatch.end_user_id`	End-user id from `X-LangWatch-End-User-Id`, its LiteLLM alias, or the `user` body field
`langwatch.reserved.request_metadata`	The `X-LangWatch-Metadata` JSON object, echoed back verbatim
`http.response.status_code`	Upstream status, set when the provider answered 400 or above
`error.type`	Error class on a failed request: `rate_limited`, `provider_timeout`, `provider_error`, `not_found`, `bad_request`, or the gateway error code that refused the call

A successful chat or messages call that returns no output, no cost and no completion tokens is dropped before export. These are the empty probe calls coding agents send, and they would otherwise fill the trace list with rows that carry no data.

Response headers

Header	Value	Set when
`X-LangWatch-Gateway-Request-Id`	`req_` plus 30 hex characters, or the `X-Request-Id` you sent	Every dispatched request
`X-LangWatch-Trace-Id`	32 hex characters	The gateway exports its own spans (`OTEL_OTLP_ENDPOINT` is set). Absent otherwise
`X-LangWatch-Span-Id`	16 hex characters	Same condition as `X-LangWatch-Trace-Id`
`Traceparent`	W3C traceparent for the span written to your project	Every dispatched request
`X-LangWatch-Gateway-Version`	Gateway version string	Every request, when the deployment sets a version
`X-LangWatch-Fallback-Count`	Number of failed attempts before the one that served the request	One or more attempts failed
`X-LangWatch-Cache-Mode`	`respect`, `force`, `disable` or `auto`	Cache control resolved a mode
`X-LangWatch-Budget-Warning`	Comma-separated `<scope>:<percent>` entries, for example `project:95`. A provider-filtered budget reads `<scope>/<modelProviderId>:<percent>`	A budget crossed its warning threshold
`X-LangWatch-Params-Dropped`	Comma-separated parameter names the parameter policy removed	A translated request dropped parameters
`X-LangWatch-Guardrails-Not-Applied`	Reason the key’s guardrails did not run, for example `realtime_session`	Guardrails were skipped
`X-LangWatch-Session-Id`	LangWatch id of the voice session a mint opened	Realtime session mints
`X-LangWatch-Provider`	Provider that returned the error	The gateway forwards a provider error
`X-Langwatch-Models-Discovery-Incomplete`	Providers that contributed no models, with the reason	`GET /v1/models` returned a partial list

Debugging a single request

Read X-LangWatch-Gateway-Request-Id off the response or the client’s error, then search that value in Trace Explorer. That span shows the model, the provider, the token counts, the cost and the upstream status.

curl -sD- -o /dev/null \
  -H "Authorization: Bearer $LANGWATCH_VIRTUAL_KEY" \
  -H "Content-Type: application/json" \
  -X POST https://gateway.langwatch.ai/v1/chat/completions \
  -d '{"model":"openai/gpt-5-mini","messages":[{"role":"user","content":"ping"}],"max_tokens":4}' \
  | grep -i '^x-langwatch-'

Trace ID propagation

Send a Traceparent request header and the span written to your project becomes a child of your trace, under your trace id. Send no header and the span starts a trace of its own. The Traceparent response header names that span. Forward it to the next hop in your system and everything stitches into one trace in Trace Explorer. X-LangWatch-Trace-Id and X-LangWatch-Span-Id name the gateway’s own operational span, which stays separate from your trace. They are present only on a deployment that exports the gateway’s own spans; LangWatch Cloud responses carry Traceparent and X-LangWatch-Gateway-Request-Id only. Use them when you report a problem to LangWatch support.

Metrics (Prometheus)

GET /metrics needs no authentication and runs on the same port as the API. Scrape it with a ServiceMonitor or a plain scrape config. See Self-hosting: Helm.

Traffic and lifecycle

Metric	Type	Labels	Meaning
`gateway_http_requests_total`	counter	`route`, `status`, `provider`, `model`	Requests served. `route` is the route pattern (`/v1/chat/completions`), `status` the HTTP status as a string
`gateway_http_request_duration_seconds`	histogram	`route`, `provider`	End-to-end latency, including auth, guardrails and the provider round trip
`gateway_in_flight_requests`	gauge	none	Requests this pod is handling now
`gateway_draining`	gauge	none	`1` while the pod drains after SIGTERM, `0` otherwise
`gateway_streaming_active`	gauge	none	Streaming responses open on this pod
`gateway_streaming_usage_missing_total`	counter	`provider`, `model`	Streams that closed without the provider reporting usage. They debit no budget, so alert on any positive rate

Provider dispatch and fallback

Metric	Type	Labels	Meaning
`gateway_provider_duration_seconds`	histogram	`provider`, `model`	Upstream round trip per attempt. Subtract it from the request duration to isolate gateway overhead
`gateway_provider_attempts_total`	counter	`credential_id`, `outcome`	One increment per dispatch attempt against a credential
`gateway_fallback_events_total`	counter	`from_credential`, `to_credential`	Failovers, by the credential that failed and the one that served the request
`gateway_circuit_state`	gauge	`credential_id`	Circuit breaker state: `0` closed, `1` open, `2` half-open

outcome is one of success, fallback_success, retryable_5xx, not_found, rate_limit, timeout, network, circuit_open, non_retryable, chain_exhausted, context_done.

Authentication cache

Metric	Type	Labels	Meaning
`gateway_auth_cache_hits_total`	counter	`tier`	Virtual-key resolutions served from cache
`gateway_auth_cache_misses_total`	counter	`tier`	Resolutions that fell through to the control plane
`gateway_auth_cache_lookups_total`	counter	none	Resolutions attempted. The denominator for the hit rate
`gateway_auth_cache_size`	gauge	`tier`	Virtual keys held in the cache

tier always reads l1. The cache is an in-process LRU, private to each pod.

Governance

Metric	Type	Labels	Meaning
`gateway_budget_blocks_total`	counter	`scope`	Requests rejected by the budget precheck, by the scope whose limit was breached
`gateway_cache_hits_total`	counter	`outcome`	Prompt-cache result: `hit` when the provider reported cache-read tokens, `miss` otherwise
`gateway_cache_rule_hits_total`	counter	`rule_id`, `mode_applied`	Cache-control rule matches. `mode_applied` is `RESPECT`, `FORCE` or `DISABLE`
`gateway_guardrail_verdicts_total`	counter	`direction`, `verdict`	Guardrail decisions. `direction` is `request`, `response` or `stream_chunk`; `verdict` is `allow`, `block`, `modify` or `fail_open`. `fail_open` means the guardrail service could not be reached and the traffic passed unchecked
`gateway_rate_limit_denied_total`	counter	`dimension`, `vk_id`	Requests denied by a rate limit. `dimension` is `rpm` or `rpd`
`gateway_client_rejects_total`	counter	`code`, `vk_id`	Requests the gateway itself rejected as the caller’s fault after authentication, by gateway error code and virtual key

gateway_client_rejects_total counts only what the gateway refused. Three cases stay off it on purpose, because each would make a per-key alert fire for a key doing its job:

Authentication failures, which have no key to attribute yet. Watch gateway_http_requests_total{status="401"}.
Provider rejections, such as an upstream 429. Watch gateway_provider_attempts_total{outcome="rate_limit"}.
Rate limiting, which a key at its ceiling would pin. Watch gateway_rate_limit_denied_total.

To find a client stuck in a rejection loop, rank the keys first:

topk(5, sum by (vk_id) (rate(gateway_client_rejects_total{code!=""}[15m])))

Then break the top key down by code, with the full key id in place of <vk-id>:

sum by (code) (rate(gateway_client_rejects_total{vk_id="<vk-id>"}[15m]))

Realtime voice

Metric	Type	Labels	Meaning
`gateway_realtime_mints_total`	counter	`vendor`, `outcome`	Voice session mint attempts. `outcome` is `minted`, `session_limit`, `registry_unavailable` or `provider_error`. `minted` counts sessions admitted at mint time; the vendor’s later report decides what a session cost
`gateway_realtime_session_limit_blocks_total`	counter	none	Mints refused because a key already held its maximum open sessions. A rise means the cap is too low for the key’s traffic, or its sessions stay open. The key is in the trace and the log line
`gateway_realtime_registry_errors_total`	counter	`operation`	Failed calls to the voice-session record, by `reserve`, `correlate`, `release` or `usage`. A `reserve` failure refuses the mint; a `correlate` failure costs the session its exact join to the vendor’s report

Control plane

Metric	Type	Labels	Meaning
`gateway_internal_rtt_seconds`	histogram	`endpoint`	Round trip to the control plane. Separates a slow control plane from a slow provider
`gateway_control_plane_requests_total`	counter	`endpoint`, `status`	Control-plane calls by outcome. `status` reads `error` when the call got no response

endpoint is one of resolve-key, guardrail-check, config, codex-refresh. The change-feed long poll is excluded, because it blocks until an event or its own timeout and its timing says little about control-plane health.

Spend emission

Every request writes a spend record to a bounded on-disk spool, and a background drainer ships the spool to the control plane. The control plane debits budgets from those records, so a record the spool drops is spend that is never billed and never enforced against.

Metric	Type	Labels	Meaning
`gateway_spend_spool_appended_total`	counter	none	Spend records written to the spool. The denominator for the drop counter
`gateway_spend_spool_dropped_total`	counter	`reason`	Records lost before they could ship

reason is intake when the spool’s writer could not take the record, and overflow when the oldest sealed segments were deleted to keep the spool inside its 64 MiB bound. Alert on any sustained positive rate:

sum by (reason) (rate(gateway_spend_spool_dropped_total[10m])) > 0

Both counters read 0 on a pod whose spool failed to open. That pod serves traffic and emits no spend at all. It logs the failure at startup, and you can spot it from a scrape by reading these two counters against a live gateway_http_requests_total. The spool is pod-local and stays off a volume. A pod deleted with records still unshipped loses them, bounded by the drain lag of about a second of traffic. The registry also carries the standard Go runtime and process collectors (go_*, process_*). Also check: Troubleshooting for reading a failed request end to end, Security for what the gateway records and what it never logs, Self-hosting: Configuration for the environment variables.

Get Started

SDK Integration

Virtual Keys & Budgets

Billing & Metering

Governance

Providers

Features

API Reference

Self-Hosting

Cookbooks

Where each signal lands

Trace attributes

Response headers

Debugging a single request

Trace ID propagation

Metrics (Prometheus)

Traffic and lifecycle

Provider dispatch and fallback

Authentication cache

Governance

Realtime voice

Control plane

Spend emission

​Where each signal lands

​Trace attributes

​Response headers

​Debugging a single request

​Trace ID propagation

​Metrics (Prometheus)

​Traffic and lifecycle

​Provider dispatch and fallback

​Authentication cache

​Governance

​Realtime voice

​Control plane

​Spend emission

Where each signal lands

Trace attributes

Response headers

Debugging a single request

Trace ID propagation

Metrics (Prometheus)

Traffic and lifecycle

Provider dispatch and fallback

Authentication cache

Governance

Realtime voice

Control plane

Spend emission