Architecture & Infrastructure

A self-hosted LangWatch deployment runs the app, the workers, PostgreSQL, Redis, ClickHouse with S3 storage, and the NLP and evaluation services, connected by an event-sourcing pipeline.

​

System Overview

​

Components

​

LangWatch App (port 5560)

Next.js server, the single external entry point for all traffic:

• Web UI: dashboards, trace explorer, prompt management, experiment views
• REST API + OTel ingestion: receives spans from LangWatch SDKs
• Authentication: email, Google, GitHub, GitLab, Azure AD, Auth0, Okta
• SSE: pushes real-time updates to connected browser clients
• Analytics queries: reads from ClickHouse
• Control plane: manages users, teams, projects via PostgreSQL

​

LangWatch Workers

Same langwatch/langwatch image, started with pnpm start:workers. Consumes jobs from a BullMQ queue in Redis and runs the event sourcing pipeline (see below).

• Deployed as a separate Kubernetes Deployment
• Stateless, scale by adding replicas

​

LangWatch NLP (port 5561)

Python service for:

• Optimization Studio workflow execution
• Topic clustering algorithms
• Custom evaluator execution

​

LangEvals (port 5562)

Built-in evaluator library (Python):

• LLM-as-a-Judge (boolean, categorical, scored)
• Safety (content safety, jailbreak detection, prompt shield)
• Quality (faithfulness, relevancy, correctness, summarization)
• RAG (context precision, context recall, context relevancy)
• Format (exact match, BLEU, ROUGE, semantic similarity)

Both NLP and LangEvals make outbound calls to external LLM providers for model-based operations.

​

PostgreSQL: Control Plane

Stores users, teams, projects, configurations, prompt versions, evaluator definitions. Managed via Prisma ORM with auto-migrations on startup.

​

ClickHouse: Data Plane

Stores all high-volume data: traces, spans, evaluations, experiments, analytics, and event sourcing events/projections. The clickhouse-serverless subchart auto-tunes internal parameters from two inputs: cpu and memory. The langwatch/clickhouse-serverless Docker image is a performance-tweaked ClickHouse build optimized for LangWatch’s traffic patterns, high-throughput event ingestion with concurrent analytical queries. Tiered storage: hot data on local SSD, cold data on S3 after a configurable TTL (default 49 days). Native BACKUP/RESTORE to S3.

​

Redis

• BullMQ job queue: connects the App to Workers with guaranteed delivery, retry, and backpressure
• Caching: frequently accessed config and lookup data
• Sessions: user session storage

​

S3, Object Storage

• ClickHouse cold storage (tiered after TTL)
• ClickHouse backups (full + incremental)
• Dataset storage (optional)
• Stored objects: externalized byte content (audio/image/video/document) content-addressed under {projectId}/{sha256} and served back via GET /api/files/:id.

​

Event Sourcing Pipeline

LangWatch v3 uses an event-sourcing model for data processing. Understanding this helps with debugging and capacity planning. When a span arrives from your SDK, it enters a pipeline of independent steps running on the Workers: Each step reads from the queue, does its work, and writes results to ClickHouse. Steps are independent, if one is slow (e.g., evaluation waiting on an LLM call), others continue processing.

​

How Data is Organized

The pipeline produces three types of output: Events: immutable records of what happened. Stored in ClickHouse and never modified. Projections: derived tables that dashboards and APIs read from. Built from events. Reactions: side effects triggered during processing.

​

Data Flow

Additionally, Kubernetes CronJobs trigger periodic tasks via HTTP on the App:

• Topic clustering: daily at midnight, via the NLP service
• Alert triggers: every 3 minutes, evaluates monitor conditions
• Retention cleanup: daily at 01:00, removes data past retention period

​

Network Topology

Only the App is exposed externally. Everything else is cluster-internal:

​

Connected agents

A connected agent is the customer’s own process holding an outbound WebSocket to the App. Three requirements:

• WebSocket upgrade on /api/v1/agents/connect. The App serves it on its normal HTTP listener, next to /api/trpc-ws. An ingress that already passes /api/trpc-ws needs no change; one that terminates or rewrites upgrades must pass this path through.
• A read timeout above 15 seconds. The App pings every 15 seconds to keep the connection alive. A shorter idle timeout closes the connection between pings, and the agent flaps between Online and Offline. The access-ingress.yaml overlay sets proxy-read-timeout: 120.
• Outbound WSS from the customer’s side. The process that runs the agent opens wss:// to your App’s public hostname. The customer’s side opens no listening port and needs no inbound rule.
• HTTP long polling for a network that blocks WebSockets. The same frames also travel over POST /api/v1/agents/connect/register, GET /api/v1/agents/connect/poll and POST /api/v1/agents/connect/frames. A poll is held open for up to 25 seconds, so the ingress read timeout must be above that; the proxy-read-timeout: 120 above covers it. The SDK switches to this path on its own when a proxy answers the upgrade with an HTTP status, or from the start with LANGWATCH_AGENT_TRANSPORT=http.

Redis is required for more than one App replica. Instance presence and call dispatch live in Redis; without it the registry is in memory, correct for one replica only, and the App refuses a connection when LANGWATCH_APP_REPLICAS is above 1. That variable defaults to 1, and Helm scales the App with app.replicaCount on its own, so a Helm deployment must set LANGWATCH_APP_REPLICAS to the same value as app.replicaCount. Without Redis, a mismatch fails in one of two ways. If app.replicaCount is above LANGWATCH_APP_REPLICAS, each pod runs its own in-memory registry, and an agent connected to one pod reads as Offline on the others. If LANGWATCH_APP_REPLICAS is above app.replicaCount, the App refuses every agent connection, also when only one pod runs.

​

Docker Images

​

OpenTelemetry Integration

LangWatch is deeply integrated with OpenTelemetry. The platform both consumes and exports telemetry data: Ingestion: The LangWatch App accepts spans via the OpenTelemetry protocol (OTLP over HTTP). Any OTel-instrumented application can send traces to LangWatch without a vendor-specific SDK. Export: LangWatch exports its own operational metrics, logs, and traces via OpenTelemetry for infrastructure debugging:

• Metrics: Prometheus-compatible metrics from the App and Workers (request latency, queue depth, error rates)
• Logs: Structured application logs from all components
• Traces: Distributed traces of internal request processing

This means you can monitor LangWatch itself using the same observability stack you use for the rest of your infrastructure, Grafana, Datadog, New Relic, or any OTel-compatible backend. LangWatch ships with off-the-shelf Grafana dashboards for monitoring the platform. See Observability & Monitoring for setup details.

​

Deployment Models

​

Self-Managed

Everything on your infrastructure. You deploy the Helm chart and manage all components.

​

Cloud Enterprise

LangWatch manages the control plane in a dedicated, single-tenant environment. Exclusive data instances in your preferred region.

​

Hybrid (Bring Your Own Storage)

LangWatch manages compute (App, Workers, NLP, LangEvals). You bring your own ClickHouse + S3 in your VPC. For Cloud Enterprise or Hybrid, contact the LangWatch team.