Control plane & integration tiers

Pairs with: AI Gateway → Overview. The gateway is the data plane (request path); this page is the control plane (configuration + fold reactors) that sits next to it in the same compose stack and Helm chart.

LangWatch is positioned as a control plane above every AI tool an enterprise uses, not just the ones whose API key path you own. The data plane (the AI Gateway proxy) is one of five integration tiers; the control plane is what stays the same across all of them. This page is the navigation map for “where does X fit?”. Use it to locate the surface you’re configuring (or asking about) on the larger ladder. Sister docs go deep on individual tiers; this one only explains the relationships.

Daily-use surfaces vs admin-authoring surfaces

The governance product UI splits into two functional layers. The distinction matters when reading the rest of this doc and the existing routes:

Surface	Purpose	Audience	Where it lives
`/governance` (forthcoming top-level home)	Daily-use org governance home: spend, ingestion-source health, anomalies, per-user breakdown. Read-mostly.	Org admins whose org’s `setupState.governanceActive` is `true` (see Persona-based home routing below)	Top-level route, org-scoped. Replaces the read-mostly content currently temporarily under `/settings/governance`.
`/me`, `/me/settings`	Daily-use personal home: your usage, your budget, your personal VK, your IDE setup snippet.	Any user with a personal VK	Top-level route, principal-scoped.
`/[project]/...`	LLMOps daily-use, traces, evals, datasets, scenarios, workflows.	Project members	Per-project routes. Unchanged from current LangWatch.
`/settings/governance/ingestion-sources`, `/settings/governance/anomaly-rules`, `/settings/routing-policies`	Admin-authoring (CRUD), create, rotate, archive sources, write rules, publish routing policies.	Org admins with `organization:manage`	Settings sub-routes, deliberately separate from daily-use surfaces.

This split is the answer to “why is governance overview hidden under Settings?”, it shouldn’t be. The read-mostly daily home is moving out to /governance (top-level); only the create, rotate, archive admin tools stay in Settings, where every other admin authoring tool lives.

Persona-based home routing: the setup-state signal

The /governance entry only appears for users whose org has activated governance. The signal is the api.governance.setupState({ organizationId }) tRPC procedure (read-only, cheap, server-side):

{
  hasPersonalVKs:       boolean,
  hasRoutingPolicies:   boolean,
  hasIngestionSources:  boolean,
  hasAnomalyRules:      boolean,
  hasRecentActivity:    boolean,  // any event in last 30d
  governanceActive:     boolean,  // OR of the above
}

The UI calls it once on session resolve. The promotion rule is deliberately conservative, don’t auto-redirect existing LLMOps users, just promote a /governance nav entry when the signal fires:

const { data } = api.governance.setupState.useQuery({ organizationId });
if (data?.governanceActive && hasOrgManagePermission) {
  // Promote /governance entry in MainMenu.
  // Otherwise: hide it. Current LLMOps customers see nothing
  // until their org actually activates governance.
}

This satisfies the “don’t lose LLMOps” constraint: the vast majority of current admins (no personal VKs, no IngestionSources, no recent gateway activity) get governanceActive=false and see no nav change at all.

The split: control plane vs data plane

┌─────────────────────────────────────────────────────────────────────┐
│                         LangWatch control plane                     │
│                                                                     │
│   Identity  ·  Policy  ·  Budgets  ·  Audit  ·  Dashboards  ·  Alert routing
│                                                                     │
│         (one set of primitives, shared by every tier below)         │
└──────┬─────────────┬─────────────┬─────────────┬─────────────┬──────┘
       │             │             │             │             │
       ▼             ▼             ▼             ▼             ▼
   ┌────────┐  ┌────────┐  ┌──────────┐  ┌──────────┐  ┌────────────┐
   │ Tier 1 │  │ Tier 2 │  │  Tier 3  │  │  Tier 4  │  │   Tier 5   │
   │Gateway │  │  BYOK  │  │   Audit  │  │   OTel   │  │  Sandbox   │
   │ proxy  │  │endpoint│  │  ingest  │  │   SDK    │  │  runtime   │
   └────────┘  └────────┘  └──────────┘  └──────────┘  └────────────┘

      ▲                                                       ▲
      │                                                       │
   most control                                          deepest control
   over LLM calls                                        over agent
   (block in-flight)                                     execution

The control-plane primitives are the same six entities described in the governance overview: Org, Members, Teams (personal + shared), Projects + IngestionSources, ApiCredentials (polymorphic owner), RoutingPolicies, Budgets. Every tier below uses them; every dashboard reads from the same trace store they emit into. The data-plane components below the control plane are independent. Customers can adopt a single tier (e.g. Tier 3 audit ingestion alone) without touching any other.

The five integration tiers

Tier	Mode	Customer owns the key path?	What you can do
1	Gateway proxy	Yes	Mid-flight inspect, rewrite, block. Mint per-user virtual keys. Enforce policy + budget before the call leaves the gateway.
2	BYOK endpoint routing	Sometimes	Customer points a SaaS platform’s “custom LLM endpoint” at the gateway. Same depth as Tier 1 for traffic that flows through it.
3	Audit log ingest	No	Pull from a closed platform’s compliance API into `IngestionSource` records. Detect anomalies, alert, trigger admin-API revoke out-of-band.
4	OTel, SDK instrumentation	Partial	Customer’s agent code (or an opt-in SaaS like Claude Cowork) emits OpenTelemetry to LangWatch. Per-turn detail, no block, but full trace context.
5	Sandboxed runtime	Yes (you host it)	LangWatch runs the agent. Egress policy, MCP allowlist, file-system jail, tool approval. Maximum control.

Every real customer is a mix. A typical large enterprise adopts: Tier 1 for Claude Code, Codex, Cursor, Gemini CLI, custom agents (BYOK paths LangWatch can proxy). Tier 3 for closed SaaS where Microsoft, Anthropic, OpenAI own the runtime (Copilot Studio, ChatGPT Enterprise, Claude for Work). Tier 4 for in-house agents already instrumented. Tier 5 later, for the agents they genuinely don’t trust.

Tier 1: Gateway proxy (where the CLI lives)

Tier 1 is everything documented under Coding CLI Integrations. The CLI’s job is to acquire a virtual key, then exec the underlying tool with the gateway’s base URL injected:

Dev terminal              LangWatch                  Provider
─────────────             ─────────                  ────────
$ langwatch               app.langwatch.ai           api.anthropic.com
  login --device  ◀──SSO──▶  /api/auth/cli/*
  ↓ writes ~/.langwatch
  ↓ ANTHROPIC_AUTH_TOKEN=vk-lw-...

$ langwatch claude  ──exec(anthropic_url)─▶  gateway.langwatch.ai
                                              │ resolve VK
                                              │ apply RoutingPolicy
                                              │ budget pre-check
                                              ▼
                                              api.anthropic.com

Why Tier 1 wins:

Per-user attribution at the credential layer. Each user’s vk-lw-... carries principal_id=<userId>. Every request, every trace, every cost roll-up attributes to the right person without parsing the model’s user field at the provider.
Mid-flight policy. Gateway can block before the call goes out (request size, model allowlist, denied tool, budget exceeded). Tier 3 only sees the call after it happened.
One credential covers every tool the dev uses. Same VK for Claude Code, Codex, Cursor, Gemini CLI, see the matrix on the CLI overview.

The CLI itself is documented end-to-end at Personal IDE keys and the per-tool config pages under /ai-gateway/cli/.

Tier 2: BYOK endpoint routing

A handful of closed SaaS platforms let an admin point their “custom LLM endpoint” at a third-party gateway. When the customer wires this up, that platform’s traffic effectively joins Tier 1: same proxy, same policy, same per-user attribution (if the SaaS forwards user identity). Confirmed BYOK paths in 2026:

Workato: Genies recipes can target a custom LLM endpoint.
Vertex AI custom endpoint: point at the gateway as a passthrough.
Bedrock proxy routing: same.
Some Copilot Studio scenarios: mileage varies; check with the customer before promising it.

When BYOK isn’t available, the platform falls back to Tier 3.

Tier 3: Audit log ingest

For closed SaaS where the runtime is owned by Anthropic, OpenAI / Microsoft, Google, Workato, LangWatch can’t proxy the call but can ingest the audit log the platform exposes. The trace pipeline writes one row per audit event into the same store as Tier 1 traces, tagged by SourceType so dashboards can union or filter at will. Per-platform mode (matching the seven IngestionSource.sourceType values shipped on this branch):

copilot_studio, Microsoft Purview pull every 1-5 min via the Office 365 Management Activity API. parserConfig: { tenantId, clientId, clientSecretHash, pollEverySec }.
claude_compliance, pull from the Claude Compliance API. parserConfig: { workspaceApiKeyHash, pollEverySec }.
claude_cowork, OTel push (Tier 4), admin pastes the LangWatch OTLP URL + bearer token into Cowork org settings; every Cowork install streams to LangWatch automatically. parserConfig: { workspaceId }.
openai_compliance, pull JSONL files at minutes-level latency from OpenAI’s compliance bucket. parserConfig: { bucket, prefix, roleArn, pollEverySec }.
workato, push to a webhook the customer configures in their Activity Audit Log streaming settings. parserConfig: { sharedSecretLastFour }.
s3_custom, generic S3 audit-log drop with a configurable parser DSL, for any platform that drops JSON, JSONL, CSV audit logs into a customer-owned bucket. parserConfig: { bucket, prefix, roleArn, parserDsl, pollEverySec }. This is the catch-all for customer-specific audit feeds that don’t have a first-party adapter yet.
otel_generic, generic OTLP/HTTP receiver. Any agent framework that speaks OpenTelemetry (Cowork OTel as a sub-case, in-house agents, OpenInference-instrumented stacks) can push to a per-source endpoint. parserConfig: { allowedSourceTypeLabel? }.

You can’t block in-flight here, but you can:

Detect (anomaly rules, “user X did N actions outside business hours”).
Alert (Slack, PagerDuty, SIEM via OCSF).
Trigger admin-API revoke: call the platform’s own admin API (Anthropic, OpenAI, Microsoft Power Platform) to disable the offending workspace key or session within seconds. Tier 3 is “near-real-time stop”, not “no stop”.

The IngestionSource entity that holds the connection metadata is shared across Tier 3, Tier 4, and the S3 audit drop case. See the admin setup doc for the per-source forms.

Tier 4: OpenTelemetry, SDK instrumentation

Customer-side agents that emit OpenInference, Traceloop, standard OTel into LangWatch. This is the deepest visibility short of running the runtime, per-turn detail, full trace context, prompt + response in the trace, all the LangWatch eval/monitoring features attach automatically. Two important Tier 4 sub-cases:

Claude Cowork OTel export. Anthropic ships Cowork with a central OTLP-endpoint config, admin pastes the LangWatch URL + bearer token in the workspace settings, every Cowork install in the org streams to LangWatch. No per-desktop config required.
In-house agents using the LangWatch SDKs: the existing langwatch SDK setup; nothing governance-specific needed.

Tier 5: Sandboxed runtime (Open Managed Agents)

For workloads the customer wants to run inside LangWatch’s own runtime: Open Managed Agents, Hermes, OpenClaw, or any custom agent. Per gateway.md the recommended 2026 stack is Firecracker microVMs with snapshot-restore for cloud, Kata Containers, Google Agent Sandbox on Kubernetes for self-hosted. Egress policy via Envoy sidecar, MCP allowlist enforced at the runtime layer. Tier 5 is post-Phase-1 work; it’s not in the current PR. Linked here so the broader story is visible.

Where this PR ships in the ladder

The current branch (PR #3524) implements Tier 1 end-to-end:

✅ CLI (langwatch login --device, langwatch claude/codex/ cursor/gemini, langwatch whoami, langwatch me, langwatch logout, langwatch init-shell, langwatch request-increase).
✅ Personal Workspace (Team + Project with isPersonal flag).
✅ Personal Virtual Keys (issued via the device-flow exchange).
✅ Routing Policies (Org/Team/Project hierarchy, provider-cred validation).
✅ /me + /me/settings dashboards (web app, gated behind the release_ui_ai_governance_enabled flag).
✅ /settings/governance admin oversight dashboard (org-admin-only via organization:manage permission, plus the governance flag). Wired to real api.activityMonitor.{summary, spendByUser, ingestionSourcesHealth, recentAnomalies} tRPC procedures; no mocked fixtures: empty data renders a setup checklist (publish a RoutingPolicy, connect an IngestionSource), data renders live cards + per-user table + source-health strip. Read-mostly daily-use surface; admin authoring lives at the /settings/governance/* sub-routes.
✅ Budget enforcement (cross-surface, gateway 402 ↔ CLI Screen-8 box ↔ web BudgetExceededBanner all derive from GatewayBudgetService.check()).
✅ Helm NOTES.txt + post-install checklist for self-hosted bootstrap.
✅ IngestionSource schema (Postgres) + trace_summaries.SourceType
- SourceId columns (ClickHouse), the data model for Tier 3
- Tier 4. Seven sourceType values supported on the schema side (see Tier 3 above).
✅ IngestionSourceService + REST receivers + tRPC admin router. POST /api/ingest/otel/:sourceId (Tier 4 entry point) and POST /api/ingest/webhook/:sourceId (Tier 3 push destination) accept events authenticated by a per-source Bearer lw_is_* secret, return 202, flip lastEventAt on the source. api.ingestionSources.{list,get,create,update, rotateSecret,archive} exposes the management surface to the admin UI; raw secret is returned exactly once on create / rotate, never echoed back afterwards.
✅ api.governance.setupState, the persona-routing signal. Read-only summary of whether the org has personal VKs, routing policies, ingestion sources, anomaly rules, or recent activity. The boolean OR (governanceActive) drives whether the /governance nav entry appears for org admins. Wires the conservative no-auto-redirect promotion rule.
✅ MainMenu Governance entry: gated on the three-of-three predicate: release_ui_ai_governance_enabled flag on AND organization:manage permission AND api.governance.setupState.governanceActive === true. Renders a “GOVERN · Preview” section between LIBRARY and AI GATEWAY · Beta with an Eye-icon link to /settings/governance. Vast majority of current LLMOps admins (no governance state) see no nav change at all.

In flight on the same branch:

🚧 IngestionSource setup forms (per-source-type, in lane B): wires to api.ingestionSources.create, surfaces the one-time secret in a “paste this into upstream” modal.
🚧 Architecture deep-dive doc (the diagram + per-tier internals).

Verified end-to-end (D2 foundation)

The IngestionSource auth path, the OTel receiver, and the lastEventAt flip were dogfooded against pnpm dev on :6660. The transcript IS the proof artifact (no UI to screenshot at this layer, that’s lane B’s setup form, in flight).

$ pnpm tsx tmp/dogfood-ingestion.ts
✓ Planted IngestionSource
  org:   123123123-NiG5AY
  id:    uotNPpD9TpASqQHTv6OXo
  name:  dogfood-otel-1777270498068
  status: awaiting_first_event
  ingestSecret: lw_is_kgOa878p…REDACTED

$ curl -sS -X POST \
    -H 'Authorization: Bearer lw_is_kgOa878p…REDACTED' \
    -d '{"resource_spans":[…]}' \
    :6660/api/ingest/otel/uotNPpD9TpASqQHTv6OXo

{"accepted":true,"bytes":68}
HTTP 202

$ pnpm tsx tmp/dogfood-ingestion.ts --check
status:        active                            ← flipped from awaiting_first_event
lastEventAt:   2026-04-27T06:15:03.361Z

Auth matrix verified in the same dogfood pass:

Authorization header	Path source-id	Expected	Observed
(none)	valid	401	401
`Bearer lw_is_unknown`	valid	401	401
`Bearer <valid>`	wrong	401	401
`Bearer <valid>`	matching	202	202
`Bearer <valid>` (replay)	matching	202 + `lastEventAt` advances monotonically	202, advanced

Web UI full loop (Lane B dogfood)

The same backend, exercised end-to-end through the admin UI: admin opens /settings/governance/ingestion-sources, fills the composer, hits Save, a one-time-reveal modal exposes the lw_is_* secret + OTLP URL, the admin copies, curls the upstream, the list page polls and flips the source from Awaiting first event → Active with “last event Ns ago”. The hero shot is the list page after the flip: IngestionSource list, Active after curl receipt

IngestionSource list, Active after curl receipt

Full 6-step carousel for reviewers:

#	Step	Shot
1	`/settings/governance/ingestion-sources`, empty state	open
2	Composer open, `sourceType` picker + dynamic per-type form	open
3	Composer filled, Generic OTel example	open
4	One-time secret reveal modal, `ingestSecret` + OTLP URL + Copy	open
5	List polled, flipped to Active, last event 13 s ago	open

The dogfood walks the full Tier-3 push contract: admin issues secret in UI → upstream curl with that secret → 202 ack → lastEventAt flips → admin sees the flip on the next poll. Secret is shown exactly once and never returned by the list/get endpoints afterwards.

CLI ingest debug (Lane A dogfood)

The same backend, exercised through the CLI without ever opening a browser. After langwatch login --device, four read-only governance debug helpers go live:

$ langwatch governance status
Governance setup state
  · Personal VKs
  · Routing policies
  ✓ Ingestion sources
  ✓ Anomaly rules
  ✓ Recent activity (30d)

Governance active: yes

$ langwatch ingest list
NAME                        TYPE          STATUS  LAST EVENT
Dogfood OTel · alexis lane  otel_generic  active  46m ago

$ langwatch ingest health JxQztBgN90QeX12-afJ0C
Dogfood OTel · alexis lane  (JxQztBgN90QeX12-afJ0C)
Status:        active
Events (24h):  1
Events (7d):   1
Events (30d):  1
Last event:    47m ago

$ langwatch ingest tail JxQztBgN90QeX12-afJ0C --limit 5
2026-04-27T07:11:18.963Z  api.call  chat_completion → claude-3-5-sonnet

Every command also accepts --json for machine output (exact contract match with the equivalent tRPC procedure, web UI render). langwatch ingest tail --follow polls every 3 s and dedupes by eventId, so the CLI and the per-source detail page show the same event stream byte-for-byte. Auth for these REST adapters reuses the same Bearer access_token the device flow persists in ~/.langwatch/config.json, no separate token. Unauthenticated calls bounce with Not logged in — run \langwatch login —device` first; unknown source IDs surface the underlying tRPC NOT_FOUNDasError: IngestionSource not found. Both error paths are asserted in specs/ai-gateway/governance/cli-ingest-debug.feature` (12 scenarios).

Admin oversight wired to real data (no mocks)

Once a source is active and events are flowing, the admin oversight dashboard at /settings/governance queries gateway_activity_events via the api.activityMonitor.* tRPC procedures. Empty data renders a setup checklist; data present renders live cards + per-user table + source-health strip.

Surface	Real-data state	Shot
`/settings/governance`, setup checklist + first real source visible in the health strip	live (`api.activityMonitor.{summary, spendByUser, ingestionSourcesHealth, recentAnomalies}`)	open
`/settings/governance/ingestion-sources`, list page after the iter-13 customer-name scrub	live (`api.ingestionSources.list`)	open
`/settings/governance/anomaly-rules`, empty list (no rules yet)	live (`api.anomalyRules.list`)	open
`/settings/governance/anomaly-rules`, composer (name + severity + ruleType + scope + JSONB threshold/destination configs)	live (`api.anomalyRules.create`)	open
`/settings/governance/anomaly-rules`, list after Create roundtrip; persisted rule grouped under Critical with the honest “rules persist now; evaluation + alerts arrive with the detection backend” banner	live (PG persistence; eval engine = Sergey Option C, deferred)	open

Tier 2, Tier 5 are deferred; the architecture doesn’t preclude them and the control-plane primitives already accommodate them.

Feature-flag gates

Governance sits behind a single fence on each surface:

Web app: the release_ui_ai_governance_enabled flag. Enabled by default on self-hosted; on SaaS, PostHog release conditions decide per organization. Existing AI Gateway menu surface stays on the unrelated release_ui_ai_gateway_menu_enabled flag, they ship independently.
CLI: always available once installed. Per-account governance entitlement is enforced server-side, so the surface ships without a client-side flag.
Backend: the tRPC procedures, REST endpoints, gateway 402 payload, and Prisma schema are not flag-gated. Hiding only the user-facing surfaces lets the long-lived branch merge cleanly without exposing preview features.

Where to next

For a dev opting in to Tier 1 today: Personal IDE keys.
For an admin onboarding a new org: Admin setup.
For the routing-policy DSL that lets one VK serve multiple tools: Routing policies.
For per-tool CLI config (env vars, dialect notes): Coding CLI Integrations.
For the architecture deep-dive once published: architecture.md (in flight on this branch).

Get Started

Personal Portal

Workspaces & Access

Dashboards

Privacy

Sources

Detection

Compliance & Architecture

Operations

Programmatic surfaces

Control plane & integration tiers

Daily-use surfaces vs admin-authoring surfaces

Persona-based home routing: the setup-state signal

The split: control plane vs data plane

The five integration tiers

Tier 1: Gateway proxy (where the CLI lives)

Tier 2: BYOK endpoint routing

Tier 3: Audit log ingest

Tier 4: OpenTelemetry, SDK instrumentation

Tier 5: Sandboxed runtime (Open Managed Agents)

Where this PR ships in the ladder

Verified end-to-end (D2 foundation)

Web UI full loop (Lane B dogfood)

CLI ingest debug (Lane A dogfood)

Admin oversight wired to real data (no mocks)

Feature-flag gates

Where to next

​Daily-use surfaces vs admin-authoring surfaces

​Persona-based home routing: the setup-state signal

​The split: control plane vs data plane

​The five integration tiers

​Tier 1: Gateway proxy (where the CLI lives)

​Tier 2: BYOK endpoint routing

​Tier 3: Audit log ingest

​Tier 4: OpenTelemetry, SDK instrumentation

​Tier 5: Sandboxed runtime (Open Managed Agents)

​Where this PR ships in the ladder

​Verified end-to-end (D2 foundation)

​Web UI full loop (Lane B dogfood)

​CLI ingest debug (Lane A dogfood)

​Admin oversight wired to real data (no mocks)

​Feature-flag gates

​Where to next

Daily-use surfaces vs admin-authoring surfaces

Persona-based home routing: the setup-state signal

The split: control plane vs data plane

The five integration tiers

Tier 1: Gateway proxy (where the CLI lives)

Tier 2: BYOK endpoint routing

Tier 3: Audit log ingest

Tier 4: OpenTelemetry, SDK instrumentation

Tier 5: Sandboxed runtime (Open Managed Agents)

Where this PR ships in the ladder

Verified end-to-end (D2 foundation)

Web UI full loop (Lane B dogfood)

CLI ingest debug (Lane A dogfood)

Admin oversight wired to real data (no mocks)

Feature-flag gates

Where to next