Start with the request id
Every gateway response carriesX-LangWatch-Gateway-Request-Id. Read it from your client’s output and search for it in Trace Explorer. The trace holds the resolved provider, the dispatch attempts, the upstream latency and the guardrail verdicts for that one request.
Error responses share one body shape:
code field is the stable identifier. Match your client code on code; the message is copy and it changes.
401 on every request
The gateway answers401 invalid_api_key when it cannot resolve the bearer token to a virtual key.
Confirm it:
401 with an empty Authorization value in your own shell means the variable never expanded in the running process. That is the most common cause.
Causes and fixes:
A revoked or disabled key answers
403, not 401. See the next section.
403 virtual_key_revoked or virtual_key_disabled
Both are deliberate stops on the key, and they are different states.
Fix: re-enable a disabled key in AI Gateway → Virtual Keys, or with
langwatch virtual-keys enable <id>. A revoked key cannot come back. Create a new key and move the callers to it.
429 with rate_limited
The virtual key hit its own requests-per-minute or requests-per-day ceiling. The gateway checks the ceiling before it calls any provider, so a rejected request burns no upstream tokens.
Confirm it: the response body carries "code":"rate_limited" and no X-LangWatch-Provider header. A 429 from the provider instead carries X-LangWatch-Provider and forwards the provider’s own Retry-After.
The response does not say which dimension fired. The metric does:
dimension is rpm or rpd.
Fix: raise or clear the ceiling on the key.
400 model_not_recognized
The gateway resolved the virtual key but could not turn the model string into a provider and a model id.
Confirm it: list what the key can actually route to.
Two neighbouring codes mean something else.
400 model_not_allowed means the model is real but the key’s allowlist refuses it. 400 model_provider_not_bound means no credential in the key’s scope serves that provider. See Model naming.
402 budget_exceeded
A budget scope on the key breached its limit. The message names the scope.
Fix: raise the limit, move the key to a scope with headroom, or change the scope’s breach action from block to warn. See Budgets.
Spend reaches the budget through the spend records the gateway ships. If the ledger is not moving at all, read Spend does not appear on the key instead.
502 or 503 during a rollout
A pod stopped serving before the load balancer removed it, or the drain window cut requests that were still running. Confirm it: during a healthy drain,gateway_draining goes to 1 on the leaving pod and gateway_in_flight_requests falls to 0 before the pod exits. A flat in-flight count past the drain window is a handler that never returned.
SIGTERM the gateway flips /readyz to 503 {"status":"draining"}, waits shutdown.preDrainWaitSeconds (default 5) so the endpoint controller removes the pod, then closes the listener and gives in-flight requests shutdown.timeoutSeconds (default 60).
Fix, 502 from the load balancer: raise shutdown.preDrainWaitSeconds so the removal has time to propagate, and raise terminationGracePeriodSeconds by the same amount. The chart requires terminationGracePeriodSeconds to be at least preDrainWaitSeconds + timeoutSeconds + 10, and refuses to render otherwise.
Fix, cut streams: a provider call can run up to 14 minutes, so the default 60 second window cuts long streams on every rollout. The gateway says so at boot with graceful_shutdown_shorter_than_max_stream_duration. To finish long streams:
preStop sleep does not help. It delays the SIGTERM, so the pod still reports ready while the sleep runs and the load balancer has no signal to act on.
A stream stops mid-way
The client received part of a response and then silence. Confirm it: the gateway never switches providers mid-stream. When an upstream stream fails after the first chunk, the gateway writes one terminal SSE frame and closes:stream_options: {"include_usage": true} on OpenAI-style requests. See Streaming.
A key change is not applied
You changed a limit, an allowlist or a credential, and requests still behave the old way. How a change reaches a pod. Each pod holds the resolved key in memory and refreshes it three ways:
Rotations, revocations, budget edits, routing changes and cache-rule edits all travel on the change feed, so 10 seconds is the normal case.
Confirm the feed is running: the pod log carries an
auth_cache_change_evict line with the change kind and the key id when a change lands. A change_feed_poll_failed line instead means the pod cannot reach the control plane, and every cached key stays as it is until the poll recovers.
Fix: restore the route to the control plane. curl http://langwatch-gateway/debug/control-plane inside the cluster prints the URL the pod resolved. A rolling restart clears every cache at once when you need the change now.
Spend does not appear on the key
Traffic is flowing, and the key’s spend and its budgets stay flat. How spend travels. The gateway writes one record per request to a spool on disk. A background drainer ships the spool to the control plane, which prices the records and writes the ledger. Budget enforcement in the gateway is a local check against the budget snapshot in the cached key, so spend only moves when the whole chain runs. Confirm it, in order:
Every dropped record is spend that is never billed and never enforced against, so alert on the drop counter.
A budget that should block but does not is the same chain seen from the other end. The precheck is local and permissive, so a stale snapshot lets requests through rather than rejecting them. It reconciles once the ledger catches up.
503 auth_upstream_unavailable
The gateway cannot reach the control plane and has no cached bundle for the key.
What still works. A key a pod has already resolved keeps serving. When its token expires while the control plane is unreachable, the pod extends the entry by LW_GATEWAY_AUTH_CACHE_SOFT_BUMP_SECONDS (default 300) on each failed refresh, up to LW_GATEWAY_AUTH_CACHE_HARD_GRACE_SECONDS past the token expiry (default 21600, six hours). Past that cap the entry is evicted and the key answers 503.
A key the pod has never seen has no cached bundle and fails at once. There is no cache warm-up on boot, so a pod added during an outage starts cold, and so does every pod after a rolling deploy.
A 401, 403 or 404 from the control plane about the key itself is not an outage. It evicts the entry immediately, with no grace.
Read the ladder in the pod log:
Fix: restore the control plane. Raise
LW_GATEWAY_AUTH_CACHE_HARD_GRACE_SECONDS while an outage is still running if you would rather extend than refuse. Set it to a negative value to switch stale serving off entirely, which is what a strict revocation policy wants.
First request to a new pod is slow
A pod resolves each virtual key from the control plane the first time it sees it, which adds one round trip. Every request after that reads the in-process cache. The cache is per pod and holds 10,000 keys. Expect the extra round trip after a scale-up and after every rolling deploy.413 payload_too_large
The request body passed the cap, which the gateway applies at the edge before authentication.
Fix: shrink the body first. A 4K image is about 6 MB once base64 encoded, so a few full-resolution images plus a long system prompt reach the 32 MiB default. Raise security.maxRequestBodyBytes in the chart only when your traffic genuinely needs it, and write it as an integer literal. Never remove the cap on a public endpoint.
The audio upload routes have their own fixed cap of 26 MB.
The gateway refuses to start
See also
- Errors: every error code and its HTTP status.
- Observability: metric names, labels and span attributes.
- Health checks: what each probe reports.
- CI smoke test: catch these failures before your callers do.