Config

The gateway reads its configuration from environment variables only. There is no config file. The Helm chart sets the variables listed on Helm chart; every other variable below keeps its default unless you add it to the pod yourself. Time spans are a plain count of seconds on a _SECONDS or _S variable: 300, never 5m. A value outside ±10 years is refused at startup.

Required

LW_GATEWAY_INTERNAL_SECRET and LW_GATEWAY_JWT_SECRET have no default: the gateway fails configuration validation and does not start when either is empty. LW_GATEWAY_BASE_URL falls back to its default instead.

Var	Purpose
`LW_GATEWAY_BASE_URL`	Base URL of the LangWatch control plane. Default `http://localhost:5560`. When neither this nor its legacy alias is set, the gateway starts on the default and logs `aigateway_control_plane_base_url_not_explicit`
`LW_GATEWAY_INTERNAL_SECRET`	HMAC secret that signs every `/api/internal/gateway/*` call and the spend ingest. The same value must be set on the control plane
`LW_GATEWAY_JWT_SECRET`	HS256 secret that verifies the JWT the control plane issues for a resolved virtual key. The same value must be set on the control plane

Server

Var	Default	Purpose
`SERVER_ADDR`	`:5563`	HTTP listen address for the API, the probes and `/metrics`
`SERVER_MAX_REQUEST_BODY_BYTES`	`33554432` (32 MiB)	Request body cap. A larger body answers `413 payload_too_large` before any provider is contacted. The `/v1/audio/transcriptions` and `/v1/speech-to-text` uploads have their own fixed cap of 26 MB
`SERVER_DRAIN_DELAY_SECONDS`	`3`	Wait between flipping `/readyz` to 503 and closing the listener. Must be zero or positive. The chart sets it from `shutdown.preDrainWaitSeconds`
`SERVER_GRACEFUL_SECONDS`	`900`	Budget for in-flight requests after the listener closes. Starts after the drain delay, so size the pod grace period above the sum of the two. Must be zero or positive. The chart sets it from `shutdown.timeoutSeconds`. Below 45 the gateway logs `graceful_shutdown_shorter_than_heartbeat_interval`; below 840 it logs `graceful_shutdown_shorter_than_max_stream_duration`
`NON_STREAMING_HEARTBEAT_INTERVAL_SECONDS`	`45`	How often a non-streaming response writes a keep-alive byte while the provider call is still running, so an edge proxy does not close an idle connection. A negative value disables the heartbeat
`ENVIRONMENT`	`local`	Environment label stamped on logs and returned as `version` by the probes. Any value other than `local` requires `BLOCK_LOCAL_HTTP_CALLS=true` and `REQUIRE_HTTPS_CUSTOM_ENDPOINTS=true`, or the gateway refuses to start

Provider endpoint security

These apply to the base URLs customers configure on their own provider credentials. Cloud metadata addresses, 0.0.0.0, :: and link-local ranges are refused in every configuration.

Var	Default	Purpose
`BLOCK_LOCAL_HTTP_CALLS`	`false`	Refuse a provider endpoint that resolves to a private, loopback or otherwise non-public address
`REQUIRE_HTTPS_CUSTOM_ENDPOINTS`	`false`	Refuse a provider endpoint with an `http://` scheme
`ALLOWED_PROXY_HOSTS`	empty	Comma-separated hostnames, matched case-insensitively, allowed to resolve to a private address while `BLOCK_LOCAL_HTTP_CALLS` is on

Auth cache

The gateway caches each resolved virtual key in process. When a cached entry passes its JWT expiry and the refresh fails for a transport reason (network error, timeout, 5xx, unparseable response), the entry keeps serving: its soft expiry is extended by SOFT_BUMP_SECONDS per failed refresh, up to the JWT expiry plus HARD_GRACE_SECONDS. A 401, 403 or 404 from the control plane evicts the entry at once.

Var	Default	Purpose
`LW_GATEWAY_AUTH_CACHE_SOFT_BUMP_SECONDS`	`300`	Extension granted per failed refresh during a control plane outage. `0` takes the default
`LW_GATEWAY_AUTH_CACHE_HARD_GRACE_SECONDS`	`21600`	Cap on stale serving, measured from the JWT expiry. `0` takes the default. A negative value disables stale serving, so a key fails at its JWT expiry
`LW_GATEWAY_AUTH_CACHE_CONFIG_TTL_SECONDS`	`60`	Age at which a cached key’s configuration (credentials, endpoints, routing) is re-fetched in the background. `0` takes the default. A negative value disables the refresh

The retired names LW_GATEWAY_AUTH_CACHE_SOFT_BUMP, LW_GATEWAY_AUTH_CACHE_HARD_GRACE and LW_GATEWAY_AUTH_CACHE_CONFIG_TTL are refused: the gateway stops at startup and names the _SECONDS variable to set instead.

Circuit breaker

Each provider credential has its own breaker. Once THRESHOLD failures land inside WINDOW_S, the breaker opens and the credential is skipped for COOLDOWN_S, after which one probe request is let through. The state is exported as gateway_circuit_state.

Var	Default	Purpose
`LW_GATEWAY_CIRCUIT_WINDOW_S`	`30`	Failure counting window
`LW_GATEWAY_CIRCUIT_THRESHOLD`	`10`	Failures inside the window that open the breaker
`LW_GATEWAY_CIRCUIT_COOLDOWN_S`	`60`	Time the breaker stays open before a probe

Spend emission

Every request writes a spend record to an on-disk spool. A background drainer ships the spool to the control plane, which debits budgets from it.

Var	Default	Purpose
`LW_GATEWAY_SPEND_ENABLED`	`true`	`false` or `0` stops emission. Budgets are then never debited
`LW_GATEWAY_SPEND_SPOOL_DIR`	`<tmp>/langwatch-gateway-spend-spool`	Spool directory. Under the chart this is inside the `/tmp` emptyDir
`LW_GATEWAY_SPEND_SPOOL_MAX_BYTES`	`67108864` (64 MiB)	Spool cap. When full, the oldest records are dropped and `gateway_spend_spool_dropped_total` increments
`LW_GATEWAY_SPEND_FLUSH_INTERVAL_SECONDS`	`1`	Longest time a record waits before it becomes shippable
`LW_GATEWAY_SPEND_INGEST_BASE_URL`	`LW_GATEWAY_BASE_URL`	Base URL of the spend ingest, when it is not the control plane

JWT secret rotation

Var	Default	Purpose
`LW_GATEWAY_JWT_SECRET_PREVIOUS`	empty	Second secret tried when a JWT fails to verify against `LW_GATEWAY_JWT_SECRET`. Set it for the 15 minutes after you change the signing secret on the control plane, then remove it

The procedure is on Helm chart. There is no previous-key variable for LW_GATEWAY_INTERNAL_SECRET.

Logging

Var	Default	Purpose
`LOG_LEVEL`	`info`	`debug`, `info`, `warn` or `error`
`LOG_FORMAT`	`json`	`json` or `pretty`
`LOG_CONSOLE_LEVEL`	`LOG_LEVEL`	Console level when `OTEL_DEBUG_COLLECTOR_ENDPOINT` is set. Ignored otherwise
`LOG_OTEL_LEVEL`	`debug`	Level shipped to the debug collector when `OTEL_DEBUG_COLLECTOR_ENDPOINT` is set. Ignored otherwise

Telemetry

These variables configure the gateway’s own spans and metrics. Customer traces go through the trace bridge below and never read OTEL_*. The exporter speaks OTLP over HTTP/protobuf only.

Var	Default	Purpose
`OTEL_EXPORTER_OTLP_ENDPOINT`	empty	Collector base URL. `/v1/traces` and `/v1/metrics` are appended. Empty turns export off
`OTEL_EXPORTER_OTLP_HEADERS`	empty	Headers on every export, `key=value,key2=value2`. Use it for a collector that needs a token
`OTEL_EXPORTER_OTLP_TRACES_ENDPOINT`	empty	Full traces URL, used as is. Wins over the base endpoint for traces
`OTEL_EXPORTER_OTLP_TRACES_HEADERS`	empty	Headers for traces. Wins over `OTEL_EXPORTER_OTLP_HEADERS`
`OTEL_EXPORTER_OTLP_PROTOCOL`	empty	Only `http/protobuf` is accepted. Any other value stops startup
`OTEL_TRACES_EXPORTER`	`otlp`	`none` turns span export off and keeps metrics
`OTEL_TRACES_SAMPLER`	`parentbased_always_on`	One of `always_on`, `always_off`, `traceidratio`, `parentbased_always_on`, `parentbased_always_off`, `parentbased_traceidratio`
`OTEL_TRACES_SAMPLER_ARG`	empty	Ratio between 0 and 1. Required by the two `traceidratio` samplers
`OTEL_SDK_DISABLED`	`false`	`true` turns every exporter off
`OTEL_EXPORTER_OTLP_METRICS_ENDPOINT`	empty	Read only to warn. Metrics always go to the base endpoint plus `/v1/metrics`
`OTEL_EXPORTER_OTLP_LOGS_ENDPOINT`	empty	Read only to warn. OTLP logs go to the debug collector only
`OTEL_DEBUG_COLLECTOR_ENDPOINT`	empty	Second collector for local development. Must resolve to this machine (`localhost`, `127.0.0.1`, `::1`, `*.localhost` or `host.docker.internal`), or startup stops
`OTEL_DEBUG_COLLECTOR_HEADERS`	empty	Headers for the debug collector

Three deprecated names still work and log a rename warning at boot: OTEL_OTLP_ENDPOINT (use OTEL_EXPORTER_OTLP_ENDPOINT), OTEL_OTLP_HEADERS (use OTEL_EXPORTER_OTLP_HEADERS) and OTEL_SAMPLE_RATIO (use OTEL_TRACES_SAMPLER=parentbased_traceidratio with OTEL_TRACES_SAMPLER_ARG). A deprecated name and its replacement set to different values stop startup. The gateway does not check the scheme on these endpoints. An http:// collector sends OTEL_EXPORTER_OTLP_HEADERS in cleartext, so use https:// whenever the collector is not on the same host. Every gateway span carries the langwatch.project_id of the virtual key, so an export pointed at a LangWatch ingest files the span under the right project. See Observability.

Customer trace bridge

Var	Default	Purpose
`CUSTOMER_TRACE_BRIDGE_BASE_URL`	`LW_GATEWAY_BASE_URL`	Base URL that receives customer traces. The gateway exports to `<base>/api/otel`
`LANGY_MIRROR_TRACE_ENDPOINT`	empty	OTLP endpoint that receives a mirror copy of spans produced for Langy virtual keys
`LANGY_MIRROR_TRACE_KEY`	empty	API key sent with the mirror export
`LANGY_MIRROR_PROJECT_ID`	empty	Project that receives the mirror copy. With all three empty the mirror is off

LANGY_MIRROR_TRACE_KEY is sent as a bearer token on every mirror export, so point LANGY_MIRROR_TRACE_ENDPOINT at an https:// URL.

Debug

Never set these on a production pod. Each one puts request content somewhere a production deployment should not have it.

Var	Purpose
`LW_LOG_MESSAGE_BODY=1`	Logs every inbound `/v1/messages` and `/v1/responses` body at `info` level, prompts included
`LW_GATEWAY_OUTBOUND_PROXY`	`http://host:port` proxy for outbound provider traffic on OpenAI, Anthropic, Gemini and Azure. This does not add anything to the gateway log, but the proxy sees every request and records what its own configuration tells it to. Bedrock uses the AWS SDK and ignores it

Legacy aliases

Deployments that predate the canonical names can keep these. The canonical name wins when both are set.

Legacy name	Canonical name
`GATEWAY_LISTEN_ADDR`	`SERVER_ADDR`
`GATEWAY_CONTROL_PLANE_URL`	`LW_GATEWAY_BASE_URL`
`GATEWAY_LOG_LEVEL`	`LOG_LEVEL`
`GATEWAY_OTEL_DEFAULT_ENDPOINT`	`OTEL_EXPORTER_OTLP_ENDPOINT`

Get Started

SDK Integration

Virtual Keys & Budgets

Billing & Metering

Governance

Providers

Features

API Reference

Self-Hosting

Cookbooks

Required

Server

Provider endpoint security

Auth cache

Circuit breaker

Spend emission

JWT secret rotation

Logging

Telemetry

Customer trace bridge

Debug

Legacy aliases

​Required

​Server

​Provider endpoint security

​Auth cache

​Circuit breaker

​Spend emission

​JWT secret rotation

​Logging

​Telemetry

​Customer trace bridge

​Debug

​Legacy aliases

Required

Server

Provider endpoint security

Auth cache

Circuit breaker

Spend emission

JWT secret rotation

Logging

Telemetry

Customer trace bridge

Debug

Legacy aliases