DNS and TLS

Give the gateway its own hostname, separate from the app. LLM clients and coding agents send requests to the gateway hostname; browsers use the app hostname. This guide covers the nginx ingress the chart renders, a cloud load balancer in front of the Service, and the checks that prove the endpoint works.

Prerequisites

A DNS zone you control, for example acme.com.
An ingress controller, or the AWS Load Balancer Controller for the NLB option below.
cert-manager with a ClusterIssuer, or a certificate you provide yourself.

Set the ingress values

The chart renders one Ingress from these values:

gateway:
  ingress:
    enabled: true
    className: nginx
    host: gateway.acme.com
    path: /v1
    pathType: Prefix
    healthPath:
      enabled: true
      path: /health
    tls:
      enabled: true
      secretName: gateway-tls
    annotations:
      cert-manager.io/cluster-issuer: letsencrypt-prod
      nginx.ingress.kubernetes.io/proxy-buffering: "off"
      nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
      nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"
      nginx.ingress.kubernetes.io/proxy-request-buffering: "off"

Replace host. The chart placeholder gateway.langwatch.ai is skipped when the control plane derives the public URL, so an install that keeps it hands CLI users no gateway URL.The four nginx.ingress.kubernetes.io/proxy-* annotations keep streamed responses unbuffered and let a stream run for an hour. Keep them.

Issue the certificate

With the cert-manager.io/cluster-issuer annotation, cert-manager requests a certificate for ingress.host and stores it in ingress.tls.secretName. A DNS-01 issuer works before the DNS record exists:

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      name: letsencrypt-prod
    solvers:
      - dns01:
          cloudflare:
            apiTokenSecretRef:
              name: cloudflare-api-token
              key: api-token

To bring your own certificate, create the TLS Secret yourself and drop the cert-manager.io/cluster-issuer annotation:

kubectl -n langwatch create secret tls gateway-tls --cert=fullchain.pem --key=privkey.pem

Create the DNS record

Point a CNAME at the ingress controller’s load balancer hostname:

kubectl -n ingress-nginx get svc ingress-nginx-controller \
  -o jsonpath='{.status.loadBalancer.ingress[0].hostname}'

Type:   CNAME
Name:   gateway
Target: <load balancer hostname>

Use a CNAME, never an A record: cloud load balancer addresses change.

Verify

dig +short gateway.acme.com
openssl s_client -connect gateway.acme.com:443 -servername gateway.acme.com </dev/null 2>&1 \
  | grep -E 'subject=|issuer='
curl -sS https://gateway.acme.com/health
curl -sS https://gateway.acme.com/v1/models -H "Authorization: Bearer $LANGWATCH_VIRTUAL_KEY"

The /health call returns {"status":"ok","checks":{"gateway":"ok","control_plane":"ok"}}. The /v1/models call returns 200 with the models your key can route to. Continue with the post-install checklist.

What the ingress publishes

Path	Match	Routes
`/v1`	`Prefix`	`/v1/chat/completions`, `/v1/messages`, `/v1/responses`, `/v1/embeddings`, `/v1/audio/`, `/v1/images/`, `/v1/models`, the realtime and ElevenLabs routes
`/health`	`Exact`, only with `ingress.healthPath.enabled: true`	The public status endpoint for uptime monitors

/healthz, /readyz, /startupz, /metrics, /debug/control-plane and /internal/* stay in-cluster. The Gemini-native routes under /v1beta are outside the /v1 prefix and are not published by the default ingress; ingress.path: / publishes every route, including the probes and /metrics.

Cloud load balancer in front of the Service

To terminate TLS on a cloud load balancer instead of the ingress, disable the ingress and turn the Service into a LoadBalancer. On AWS with the Load Balancer Controller and an ACM certificate:

gateway:
  ingress:
    enabled: false
  publicUrl: https://gateway.acme.com   # no ingress host to derive it from
  service:
    type: LoadBalancer
    port: 443
    targetPort: 5563
    annotations:
      service.beta.kubernetes.io/aws-load-balancer-type: nlb
      service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
      service.beta.kubernetes.io/aws-load-balancer-ssl-cert: arn:aws:acm:<region>:<account>:certificate/<id>
      service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "443"
      service.beta.kubernetes.io/aws-load-balancer-backend-protocol: tcp

Both examples terminate TLS in front of the gateway and reach port 5563 over plain HTTP, so the Authorization header that carries the virtual key crosses that last hop without encryption. The ingress terminates at the ingress controller and the network load balancer terminates at the load balancer, which puts the plaintext hop inside the cluster network in the first case and between the load balancer and the pods in the second. Both assume that network is trusted. Where it is not, run a service mesh with mutual TLS between the terminator and the gateway pods.

Request the certificate first and publish the validation CNAME records ACM prints:

aws acm request-certificate --domain-name gateway.acme.com --validation-method DNS --region <region>
aws acm describe-certificate --certificate-arn <arn> --region <region> \
  --query 'Certificate.DomainValidationOptions[].ResourceRecord'

Then read the load balancer hostname and point the CNAME at it:

kubectl -n langwatch get svc langwatch-gateway \
  -o jsonpath='{.status.loadBalancer.ingress[0].hostname}'

A Service of type LoadBalancer publishes every route, including /metrics, the probes and /debug/control-plane, which returns the resolved LW_GATEWAY_BASE_URL and takes no key. Put a load balancer listener rule or a security group in front of it when the address is public. Set service.internal.enabled: true to also render a ClusterIP Service named langwatch-gateway-internal for in-cluster callers, so they do not go out through the load balancer.

Other ingress controllers

Set ingress.className and replace the nginx annotations with the controller’s equivalents. Two settings matter for every controller: response buffering off, and an idle or read timeout above the longest stream you allow. The gateway lets a provider call run up to 14 minutes. AWS ALB example:

gateway:
  ingress:
    className: alb
    annotations:
      alb.ingress.kubernetes.io/scheme: internet-facing
      alb.ingress.kubernetes.io/target-type: ip
      alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS":443}]'
      alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:<region>:<account>:certificate/<id>
      alb.ingress.kubernetes.io/load-balancer-attributes: idle_timeout.timeout_seconds=3600

Outbound endpoint policy

TLS in front of the gateway is one boundary. The other is the provider endpoints that customers configure on their own credentials. Three variables, set through gateway.security.* in the chart, control them:

Chart value	Env var	Effect
`security.requireHTTPSCustomEndpoints`	`REQUIRE_HTTPS_CUSTOM_ENDPOINTS`	Refuse a provider base URL with an `http://` scheme
`security.blockLocalHTTPCalls`	`BLOCK_LOCAL_HTTP_CALLS`	Refuse a provider base URL that resolves to a private, loopback or other non-public address
`security.allowedProxyHosts`	`ALLOWED_PROXY_HOSTS`	Hostnames that may resolve to a private address while the block is on

Both switches default to false, so a self-hosted install can route to a model server on the cluster network. Cloud metadata addresses, 0.0.0.0, :: and link-local ranges are refused with every setting. Setting ENVIRONMENT to anything other than local requires both switches to be true.

Common failures

Symptom	Cause	Fix
`curl` reports an unknown certificate	cert-manager has not issued `gateway-tls` yet	`kubectl -n langwatch describe certificate gateway-tls` shows the pending challenge
Streamed responses arrive all at once	Buffering is on at the proxy	Add the `proxy-buffering: "off"` annotation, or its equivalent on your controller
A long stream stops at 60 seconds	The load balancer idle timeout is shorter than the stream	Raise the controller’s idle or read timeout
CLI users get `http://localhost:5563` as the gateway URL	The control plane has no `LW_GATEWAY_PUBLIC_URL`	Set `gateway.ingress.host` to your hostname, or `gateway.publicUrl`
A provider credential with a private base URL fails to dispatch	`BLOCK_LOCAL_HTTP_CALLS` is on	Add the hostname to `ALLOWED_PROXY_HOSTS`

More symptoms are on Troubleshooting.

Get Started

SDK Integration

Virtual Keys & Budgets

Billing & Metering

Governance

Providers

Features

API Reference

Self-Hosting

Cookbooks

Prerequisites

What the ingress publishes

Cloud load balancer in front of the Service

Other ingress controllers

Outbound endpoint policy

Common failures

​Prerequisites

​What the ingress publishes

​Cloud load balancer in front of the Service

​Other ingress controllers

​Outbound endpoint policy

​Common failures

Prerequisites

What the ingress publishes

Cloud load balancer in front of the Service

Other ingress controllers

Outbound endpoint policy

Common failures