> ## Documentation Index
> Fetch the complete documentation index at: https://langwatch.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# CI smoke test end-to-end gateway health check

> Provision a short-lived VK, make a real call, verify the response, then revoke. Runs in under 10 seconds.

Goal: catch misconfigurations in the gateway BEFORE they surface during an on-call incident. This recipe provisions a temporary virtual key, calls `/v1/chat/completions`, verifies the response and its trace headers, and cleans up, all from a CI job using only the `langwatch` CLI and `curl`.

<Info>Runtime: typically 3–8 seconds wall-clock. One gateway request + two REST round-trips.</Info>

## What it validates

* The public REST API at `/api/gateway/v1/*` is reachable with the CI token.
* A ModelProvider scoped at ORG/TEAM/PROJECT is healthy (reaches actual upstream) via its Advanced (Gateway) tab settings.
* VK creation + secret reveal flow works at the requested scope.
* VK resolves end-to-end against the gateway data plane.
* Response carries `X-LangWatch-Trace-Id`, `X-LangWatch-Span-Id`, `X-LangWatch-Request-Id`, and `traceparent`.
* VK revocation propagates (best-effort, cache TTL applies).

## Prerequisites

* A CI-scoped API token stored in `CI_LANGWATCH_TOKEN` with scopes `virtualKeys:create`, `virtualKeys:delete`, `modelProviders:view` (all at the same ORGANIZATION the CI VK is created in).
* A `CI_ORG_SLUG` env var naming the org scope (e.g. `acme`). The CI VK lives at this ORG scope so its eligible-MP set is the union of every ModelProvider visible from the org.
* At least one healthy ModelProvider configured under **Settings → Model Providers** at ORG, TEAM, or PROJECT scope. Set the rate-limit / fallback fields on its Advanced (Gateway) tab if you want CI to exercise them.
* `jq` available on the runner.

## The script

```bash theme={null}
#!/usr/bin/env bash
# ci-smoke-test.sh
set -euo pipefail

export LANGWATCH_API_KEY="$CI_LANGWATCH_TOKEN"
GATEWAY_URL="${LANGWATCH_GATEWAY_URL:-https://gateway.langwatch.ai}"
TIMESTAMP=$(date -u +%Y%m%dT%H%M%S)

cleanup() {
  if [ -n "${VK_ID:-}" ]; then
    echo "Revoking $VK_ID..."
    langwatch virtual-keys revoke "$VK_ID" --format json >/dev/null || true
  fi
}
trap cleanup EXIT

# 1. Mint short-lived VK at the org scope (uses the org's default RoutingPolicy)
echo "Creating CI VK at ORG:$CI_ORG_SLUG..."
CREATE=$(langwatch virtual-keys create \
  --name "ci-smoke-$TIMESTAMP" \
  --env test \
  --scope "ORG:$CI_ORG_SLUG" \
  --format json)

VK_ID=$(echo "$CREATE" | jq -r '.virtual_key.id')
VK_SECRET=$(echo "$CREATE" | jq -r '.secret')

[ -z "$VK_ID" ] && { echo "FAIL: empty VK id"; exit 1; }
[ -z "$VK_SECRET" ] && { echo "FAIL: empty secret"; exit 1; }

# 2. Call the gateway
echo "Calling $GATEWAY_URL/v1/chat/completions..."
TRACEPARENT="00-$(openssl rand -hex 16)-$(openssl rand -hex 8)-01"

RESPONSE_FILE=$(mktemp)
HEADERS_FILE=$(mktemp)
curl --fail-with-body --max-time 10 \
  -H "Authorization: Bearer $VK_SECRET" \
  -H "Content-Type: application/json" \
  -H "traceparent: $TRACEPARENT" \
  -D "$HEADERS_FILE" -o "$RESPONSE_FILE" \
  -X POST "$GATEWAY_URL/v1/chat/completions" \
  -d '{"model":"gpt-5-mini","messages":[{"role":"user","content":"ping"}],"max_tokens":4}'

# 3. Verify headers
REQUEST_ID=$(grep -i '^x-langwatch-request-id:' "$HEADERS_FILE" | awk '{print $2}' | tr -d '\r')
RESP_TRACE_ID=$(grep -i '^x-langwatch-trace-id:' "$HEADERS_FILE" | awk '{print $2}' | tr -d '\r')
RESP_SPAN_ID=$(grep -i '^x-langwatch-span-id:' "$HEADERS_FILE" | awk '{print $2}' | tr -d '\r')
RESP_TRACEPARENT=$(grep -i '^traceparent:' "$HEADERS_FILE" | awk '{print $2}' | tr -d '\r')

[ -z "$REQUEST_ID" ] && { echo "FAIL: missing X-LangWatch-Request-Id"; exit 1; }
[ ${#RESP_TRACE_ID} -ne 32 ] && { echo "FAIL: trace-id not 32 hex ($RESP_TRACE_ID)"; exit 1; }
[ ${#RESP_SPAN_ID} -ne 16 ] && { echo "FAIL: span-id not 16 hex ($RESP_SPAN_ID)"; exit 1; }

# Trace id should match the incoming traceparent's trace id (chars 4..36)
EXPECTED_TRACE_ID=${TRACEPARENT:3:32}
if [ "$RESP_TRACE_ID" != "$EXPECTED_TRACE_ID" ]; then
  echo "FAIL: trace propagation — expected $EXPECTED_TRACE_ID, got $RESP_TRACE_ID"
  exit 1
fi

# Response traceparent should carry the same trace-id onward
if [[ "$RESP_TRACEPARENT" != "00-$EXPECTED_TRACE_ID-"* ]]; then
  echo "FAIL: response traceparent does not carry incoming trace-id"
  exit 1
fi

# 4. Verify payload
CONTENT=$(jq -r '.choices[0].message.content' "$RESPONSE_FILE")
[ -z "$CONTENT" ] || [ "$CONTENT" = "null" ] && { echo "FAIL: empty content"; exit 1; }

echo "OK — smoke test passed (request_id=$REQUEST_ID, trace=$RESP_TRACE_ID)"
```

## GitHub Actions

```yaml theme={null}
# .github/workflows/gateway-smoke.yml
name: Gateway smoke test
on:
  schedule:
    - cron: '*/15 * * * *'   # every 15 minutes
  workflow_dispatch:

jobs:
  smoke:
    runs-on: ubuntu-latest
    timeout-minutes: 2
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: 20 }
      - run: npm i -g langwatch
      - run: bash ci-smoke-test.sh
        env:
          CI_LANGWATCH_TOKEN: ${{ secrets.CI_LANGWATCH_TOKEN }}
          CI_ORG_SLUG:         ${{ vars.CI_ORG_SLUG }}
          LANGWATCH_GATEWAY_URL: https://gateway.langwatch.ai
```

## Interpreting failures

| Failure                          | What's broken                                                         | Where to look                                                                                                                           |
| -------------------------------- | --------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------- |
| `FAIL: empty VK id`              | REST API returning malformed response                                 | Check `GET /api/gateway/v1/virtual-keys` manually; examine control-plane logs                                                           |
| `curl ... exit 7`                | Gateway unreachable                                                   | Check the LB + DNS; `nslookup gateway.langwatch.ai`                                                                                     |
| `curl ... 401`                   | VK secret didn't resolve at the gateway                               | Auth cache bootstrapped against a stale `/changes` feed; check control-plane-to-gateway HMAC agreement                                  |
| `curl ... 403 model_not_allowed` | No ModelProvider visible from `ORG:$CI_ORG_SLUG` exposes `gpt-5-mini` | Replace the model in the script with one your eligible ModelProviders actually support, or add an OpenAI ModelProvider at the org scope |
| `FAIL: trace propagation`        | Per-tenant OTel middleware is off                                     | See [Observability → trace-id handshake](/docs/ai-gateway/observability#trace-id-propagation--concrete-handshake)                            |
| `FAIL: empty content`            | Upstream returned no content, provider outage                         | Compare to raw response body saved in `$RESPONSE_FILE`                                                                                  |

## Running it against self-hosted

```bash theme={null}
export LANGWATCH_ENDPOINT="https://langwatch.your-corp.internal"
export LANGWATCH_GATEWAY_URL="https://gateway.your-corp.internal"
bash ci-smoke-test.sh
```

Both URLs can be the same hostname if your LB routes `/api/*` to the control plane and `/v1/*` to the gateway fleet.

## Alerting

Pipe the script output into your alerting channel on failure. With PagerDuty:

```yaml theme={null}
      - name: Alert on failure
        if: failure()
        run: |
          curl -X POST https://events.pagerduty.com/v2/enqueue \
            -d '{"routing_key":"${{ secrets.PD_KEY }}","event_action":"trigger","payload":{"summary":"Gateway smoke test failed","severity":"error","source":"github-actions"}}'
```

Run this test at a higher frequency than your `/readyz` scrapes, it exercises the full hot path (auth cache → bifrost → upstream → debit enqueue → trace export) whereas `/readyz` only checks internal state.

## See also

* [langwatch CLI](/docs/ai-gateway/cli/langwatch-cli): the `virtual-keys` commands used here.
* [Observability](/docs/ai-gateway/observability): what each response header means.
* [Health Checks](/docs/ai-gateway/self-hosting/health-checks): complementary internal-state probe.