Skip to main content

Documentation Index

Fetch the complete documentation index at: https://langwatch.ai/docs/llms.txt

Use this file to discover all available pages before exploring further.

Goal: catch misconfigurations in the gateway BEFORE they surface during an on-call incident. This recipe provisions a temporary virtual key, calls /v1/chat/completions, verifies the response and its trace headers, and cleans up — all from a CI job using only the langwatch CLI and curl.
Runtime: typically 3–8 seconds wall-clock. One gateway request + two REST round-trips.

​
What it validates

  • The public REST API at /api/gateway/v1/* is reachable with the CI token.
  • Your gpc_* provider binding is healthy (reach actual upstream).
  • VK creation + secret reveal flow works.
  • VK resolves end-to-end against the gateway data plane.
  • Response carries X-LangWatch-Trace-Id, X-LangWatch-Span-Id, X-LangWatch-Request-Id, and traceparent.
  • VK revocation propagates (best-effort — cache TTL applies).

​
Prerequisites

  • A CI-scoped API token stored in CI_LANGWATCH_TOKEN with scopes virtualKeys:create, virtualKeys:delete, gatewayProviders:view.
  • A healthy provider binding id in CI_GPC_ID (e.g. gpc_01HZX...). Mint it manually once in the LangWatch UI → Gateway → Providers or via langwatch gateway-providers create.
  • jq available on the runner.

​
The script

#!/usr/bin/env bash
# ci-smoke-test.sh
set -euo pipefail

export LANGWATCH_API_KEY="$CI_LANGWATCH_TOKEN"
GATEWAY_URL="${LANGWATCH_GATEWAY_URL:-https://gateway.langwatch.ai}"
TIMESTAMP=$(date -u +%Y%m%dT%H%M%S)

cleanup() {
  if [ -n "${VK_ID:-}" ]; then
    echo "Revoking $VK_ID..."
    langwatch virtual-keys revoke "$VK_ID" --format json >/dev/null || true
  fi
}
trap cleanup EXIT

# 1. Mint short-lived VK
echo "Creating CI VK..."
CREATE=$(langwatch virtual-keys create \
  --name "ci-smoke-$TIMESTAMP" \
  --env test \
  --provider "$CI_GPC_ID" \
  --format json)

VK_ID=$(echo "$CREATE" | jq -r '.virtual_key.id')
VK_SECRET=$(echo "$CREATE" | jq -r '.secret')

[ -z "$VK_ID" ] && { echo "FAIL: empty VK id"; exit 1; }
[ -z "$VK_SECRET" ] && { echo "FAIL: empty secret"; exit 1; }

# 2. Call the gateway
echo "Calling $GATEWAY_URL/v1/chat/completions..."
TRACEPARENT="00-$(openssl rand -hex 16)-$(openssl rand -hex 8)-01"

RESPONSE_FILE=$(mktemp)
HEADERS_FILE=$(mktemp)
curl --fail-with-body --max-time 10 \
  -H "Authorization: Bearer $VK_SECRET" \
  -H "Content-Type: application/json" \
  -H "traceparent: $TRACEPARENT" \
  -D "$HEADERS_FILE" -o "$RESPONSE_FILE" \
  -X POST "$GATEWAY_URL/v1/chat/completions" \
  -d '{"model":"gpt-5-mini","messages":[{"role":"user","content":"ping"}],"max_tokens":4}'

# 3. Verify headers
REQUEST_ID=$(grep -i '^x-langwatch-request-id:' "$HEADERS_FILE" | awk '{print $2}' | tr -d '\r')
RESP_TRACE_ID=$(grep -i '^x-langwatch-trace-id:' "$HEADERS_FILE" | awk '{print $2}' | tr -d '\r')
RESP_SPAN_ID=$(grep -i '^x-langwatch-span-id:' "$HEADERS_FILE" | awk '{print $2}' | tr -d '\r')
RESP_TRACEPARENT=$(grep -i '^traceparent:' "$HEADERS_FILE" | awk '{print $2}' | tr -d '\r')

[ -z "$REQUEST_ID" ] && { echo "FAIL: missing X-LangWatch-Request-Id"; exit 1; }
[ ${#RESP_TRACE_ID} -ne 32 ] && { echo "FAIL: trace-id not 32 hex ($RESP_TRACE_ID)"; exit 1; }
[ ${#RESP_SPAN_ID} -ne 16 ] && { echo "FAIL: span-id not 16 hex ($RESP_SPAN_ID)"; exit 1; }

# Trace id should match the incoming traceparent's trace id (chars 4..36)
EXPECTED_TRACE_ID=${TRACEPARENT:3:32}
if [ "$RESP_TRACE_ID" != "$EXPECTED_TRACE_ID" ]; then
  echo "FAIL: trace propagation — expected $EXPECTED_TRACE_ID, got $RESP_TRACE_ID"
  exit 1
fi

# Response traceparent should carry the same trace-id onward
if [[ "$RESP_TRACEPARENT" != "00-$EXPECTED_TRACE_ID-"* ]]; then
  echo "FAIL: response traceparent does not carry incoming trace-id"
  exit 1
fi

# 4. Verify payload
CONTENT=$(jq -r '.choices[0].message.content' "$RESPONSE_FILE")
[ -z "$CONTENT" ] || [ "$CONTENT" = "null" ] && { echo "FAIL: empty content"; exit 1; }

echo "OK — smoke test passed (request_id=$REQUEST_ID, trace=$RESP_TRACE_ID)"

​
GitHub Actions

# .github/workflows/gateway-smoke.yml
name: Gateway smoke test
on:
  schedule:
    - cron: '*/15 * * * *'   # every 15 minutes
  workflow_dispatch:

jobs:
  smoke:
    runs-on: ubuntu-latest
    timeout-minutes: 2
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: 20 }
      - run: npm i -g langwatch
      - run: bash ci-smoke-test.sh
        env:
          CI_LANGWATCH_TOKEN: ${{ secrets.CI_LANGWATCH_TOKEN }}
          CI_GPC_ID:           ${{ vars.CI_GPC_ID }}
          LANGWATCH_GATEWAY_URL: https://gateway.langwatch.ai

​
Interpreting failures

FailureWhat’s brokenWhere to look
FAIL: empty VK idREST API returning malformed responseCheck GET /api/gateway/v1/virtual-keys manually; examine control-plane logs
curl ... exit 7Gateway unreachableCheck the LB + DNS; nslookup gateway.langwatch.ai
curl ... 401VK secret didn’t resolve at the gatewayAuth cache bootstrapped against a stale /changes feed; check control-plane-to-gateway HMAC agreement
curl ... 403 model_not_allowedVK’s provider binding doesn’t expose gpt-5-miniReplace the model in the script with one your $CI_GPC_ID actually supports
FAIL: trace propagationPer-tenant OTel middleware is offSee Observability → trace-id handshake
FAIL: empty contentUpstream returned no content — provider outageCompare to raw response body saved in $RESPONSE_FILE

​
Running it against self-hosted

export LANGWATCH_ENDPOINT="https://langwatch.your-corp.internal"
export LANGWATCH_GATEWAY_URL="https://gateway.your-corp.internal"
bash ci-smoke-test.sh
Both URLs can be the same hostname if your LB routes /api/* to the control plane and /v1/* to the gateway fleet.

​
Alerting

Pipe the script output into your alerting channel on failure. With PagerDuty: 
      - name: Alert on failure
        if: failure()
        run: |
          curl -X POST https://events.pagerduty.com/v2/enqueue \
            -d '{"routing_key":"${{ secrets.PD_KEY }}","event_action":"trigger","payload":{"summary":"Gateway smoke test failed","severity":"error","source":"github-actions"}}'
Run this test at a higher frequency than your /readyz scrapes — it exercises the full hot path (auth cache → bifrost → upstream → debit enqueue → trace export) whereas /readyz only checks internal state.

​
See also