> ## Documentation Index
> Fetch the complete documentation index at: https://langwatch.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.
# Control plane & integration tiers
> Where the LangWatch CLI, AI Gateway, web app, and Activity Monitor fit in one shared control plane, and how the five integration tiers stack up.
**Pairs with:** [AI Gateway → Overview](/docs/ai-gateway/overview). The gateway is the data plane (request path); this page is the control plane (configuration + fold reactors) that sits next to it in the same compose stack and Helm chart.
LangWatch is positioned as a **control plane** above every AI tool an
enterprise uses, not just the ones whose API key path you own. The
data plane (the AI Gateway proxy) is one of five integration tiers; the
control plane is what stays the same across all of them.
This page is the navigation map for "where does X fit?". Use it to
locate the surface you're configuring (or asking about) on the larger
ladder. Sister docs go deep on individual tiers; this one only
explains the relationships.
## Daily-use surfaces vs admin-authoring surfaces
The governance product UI splits into two functional layers. The
distinction matters when reading the rest of this doc and the
existing routes:
| Surface | Purpose | Audience | Where it lives |
| ------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------- |
| `/governance` (forthcoming top-level home) | Daily-use **org governance home**: spend, ingestion-source health, anomalies, per-user breakdown. Read-mostly. | Org admins whose org's `setupState.governanceActive` is `true` (see [Persona-based home routing](#persona-based-home-routing--the-setup-state-signal) below) | Top-level route, org-scoped. Replaces the read-mostly content currently temporarily under `/settings/governance`. |
| `/me`, `/me/settings` | Daily-use **personal home**: your usage, your budget, your personal VK, your IDE setup snippet. | Any user with a personal VK | Top-level route, principal-scoped. |
| `/[project]/...` | LLMOps daily-use, traces, evals, datasets, scenarios, workflows. | Project members | Per-project routes. **Unchanged** from current LangWatch. |
| `/settings/governance/ingestion-sources`, `/settings/governance/anomaly-rules`, `/settings/routing-policies` | Admin-authoring (CRUD), create, rotate, archive sources, write rules, publish routing policies. | Org admins with `organization:manage` | Settings sub-routes, deliberately separate from daily-use surfaces. |
This split is the answer to "why is governance overview hidden under
Settings?", it shouldn't be. The read-mostly daily home is moving
out to `/governance` (top-level); only the create, rotate, archive
admin tools stay in Settings, where every other admin authoring
tool lives.
### Persona-based home routing: the setup-state signal
The `/governance` entry only appears for users whose org has
**activated** governance. The signal is the
`api.governance.setupState({ organizationId })` tRPC procedure
(read-only, cheap, server-side):
```ts theme={null}
{
hasPersonalVKs: boolean,
hasRoutingPolicies: boolean,
hasIngestionSources: boolean,
hasAnomalyRules: boolean,
hasRecentActivity: boolean, // any event in last 30d
governanceActive: boolean, // OR of the above
}
```
The UI calls it once on session resolve. The promotion rule is
deliberately conservative, **don't auto-redirect existing
LLMOps users**, just promote a `/governance` nav entry when
the signal fires:
```tsx theme={null}
const { data } = api.governance.setupState.useQuery({ organizationId });
if (data?.governanceActive && hasOrgManagePermission) {
// Promote /governance entry in MainMenu.
// Otherwise: hide it. Current LLMOps customers see nothing
// until their org actually activates governance.
}
```
This satisfies the "don't lose LLMOps" constraint: the vast
majority of current admins (no personal VKs, no IngestionSources,
no recent gateway activity) get `governanceActive=false` and see
no nav change at all.
## The split: control plane vs data plane
```
┌─────────────────────────────────────────────────────────────────────┐
│ LangWatch control plane │
│ │
│ Identity · Policy · Budgets · Audit · Dashboards · Alert routing
│ │
│ (one set of primitives, shared by every tier below) │
└──────┬─────────────┬─────────────┬─────────────┬─────────────┬──────┘
│ │ │ │ │
▼ ▼ ▼ ▼ ▼
┌────────┐ ┌────────┐ ┌──────────┐ ┌──────────┐ ┌────────────┐
│ Tier 1 │ │ Tier 2 │ │ Tier 3 │ │ Tier 4 │ │ Tier 5 │
│Gateway │ │ BYOK │ │ Audit │ │ OTel │ │ Sandbox │
│ proxy │ │endpoint│ │ ingest │ │ SDK │ │ runtime │
└────────┘ └────────┘ └──────────┘ └──────────┘ └────────────┘
▲ ▲
│ │
most control deepest control
over LLM calls over agent
(block in-flight) execution
```
The control-plane primitives are the same six entities described in
the [governance overview](/docs/ai-gateway/governance/overview): Org,
Members, Teams (personal + shared), Projects + IngestionSources,
ApiCredentials (polymorphic owner), RoutingPolicies, Budgets. Every
tier below uses them; every dashboard reads from the same trace store
they emit into.
The data-plane components below the control plane are independent.
Customers can adopt a single tier (e.g. Tier 3 audit ingestion alone)
without touching any other.
## The five integration tiers
| Tier | Mode | Customer owns the key path? | What you can do |
| ----- | ------------------------- | --------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **1** | Gateway proxy | Yes | Mid-flight inspect, rewrite, **block**. Mint per-user virtual keys. Enforce policy + budget before the call leaves the gateway. |
| **2** | BYOK endpoint routing | Sometimes | Customer points a SaaS platform's "custom LLM endpoint" at the gateway. Same depth as Tier 1 *for traffic that flows through it*. |
| **3** | Audit log ingest | No | Pull from a closed platform's compliance API into [`IngestionSource`](#tier-3--audit-log-ingest) records. Detect anomalies, alert, **trigger admin-API revoke** out-of-band. |
| **4** | OTel, SDK instrumentation | Partial | Customer's agent code (or an opt-in SaaS like Claude Cowork) emits OpenTelemetry to LangWatch. Per-turn detail, no block, but full trace context. |
| **5** | Sandboxed runtime | Yes (you host it) | LangWatch runs the agent. Egress policy, MCP allowlist, file-system jail, tool approval. Maximum control. |
Every real customer is a mix. A typical large enterprise adopts:
**Tier 1** for Claude Code, Codex, Cursor, Gemini CLI, custom
agents (BYOK paths LangWatch can proxy). **Tier 3** for closed SaaS
where Microsoft, Anthropic, OpenAI own the runtime (Copilot Studio,
ChatGPT Enterprise, Claude for Work). **Tier 4** for in-house agents
already instrumented. **Tier 5** later, for the agents they
genuinely don't trust.
## Tier 1: Gateway proxy (where the CLI lives)
Tier 1 is everything documented under
[Coding CLI Integrations](/docs/ai-gateway/cli/overview). The CLI's job
is to acquire a virtual key, then `exec` the underlying tool with
the gateway's base URL injected:
```
Dev terminal LangWatch Provider
───────────── ───────── ────────
$ langwatch app.langwatch.ai api.anthropic.com
login --device ◀──SSO──▶ /api/auth/cli/*
↓ writes ~/.langwatch
↓ ANTHROPIC_AUTH_TOKEN=vk-lw-...
$ langwatch claude ──exec(anthropic_url)─▶ gateway.langwatch.ai
│ resolve VK
│ apply RoutingPolicy
│ budget pre-check
▼
api.anthropic.com
```
Why Tier 1 wins:
* **Per-user attribution at the credential layer.** Each user's
`vk-lw-...` carries `principal_id=`. Every request,
every trace, every cost roll-up attributes to the right person
without parsing the model's `user` field at the provider.
* **Mid-flight policy.** Gateway can block before the call goes
out (request size, model allowlist, denied tool, budget
exceeded). Tier 3 only sees the call after it happened.
* **One credential covers every tool the dev uses.** Same VK for
Claude Code, Codex, Cursor, Gemini CLI, see the matrix on the
[CLI overview](/docs/ai-gateway/cli/overview).
The CLI itself is documented end-to-end at
[Personal IDE keys](/docs/ai-gateway/governance/personal-keys) and
the per-tool config pages under `/ai-gateway/cli/`.
## Tier 2: BYOK endpoint routing
A handful of closed SaaS platforms let an admin point their "custom
LLM endpoint" at a third-party gateway. When the customer wires this
up, that platform's traffic effectively joins Tier 1: same proxy,
same policy, same per-user attribution (if the SaaS forwards user
identity).
Confirmed BYOK paths in 2026:
* **Workato**: Genies recipes can target a custom LLM endpoint.
* **Vertex AI custom endpoint**: point at the gateway as a
passthrough.
* **Bedrock proxy routing**: same.
* **Some Copilot Studio scenarios**: mileage varies; check with
the customer before promising it.
When BYOK isn't available, the platform falls back to Tier 3.
## Tier 3: Audit log ingest
For closed SaaS where the runtime is owned by Anthropic, OpenAI /
Microsoft, Google, Workato, LangWatch can't proxy the call but
can ingest the **audit log** the platform exposes. The trace pipeline
writes one row per audit event into the same store as Tier 1 traces,
tagged by `SourceType` so dashboards can union or filter at will.
Per-platform mode (matching the seven `IngestionSource.sourceType`
values shipped on this branch):
* **`copilot_studio`**, Microsoft Purview pull every 1-5 min via
the Office 365 Management Activity API. `parserConfig`:
`{ tenantId, clientId, clientSecretHash, pollEverySec }`.
* **`claude_compliance`**, pull from the Claude Compliance API.
`parserConfig`: `{ workspaceApiKeyHash, pollEverySec }`.
* **`claude_cowork`**, OTel push (Tier 4), admin pastes the
LangWatch OTLP URL + bearer token into Cowork org settings;
every Cowork install streams to LangWatch automatically.
`parserConfig`: `{ workspaceId }`.
* **`openai_compliance`**, pull JSONL files at minutes-level
latency from OpenAI's compliance bucket. `parserConfig`:
`{ bucket, prefix, roleArn, pollEverySec }`.
* **`workato`**, push to a webhook the customer configures
in their Activity Audit Log streaming settings.
`parserConfig`: `{ sharedSecretLastFour }`.
* **`s3_custom`**, generic S3 audit-log drop with a configurable
parser DSL, for any platform that drops JSON, JSONL, CSV
audit logs into a customer-owned bucket. `parserConfig`:
`{ bucket, prefix, roleArn, parserDsl, pollEverySec }`.
This is the catch-all for customer-specific audit feeds that
don't have a first-party adapter yet.
* **`otel_generic`**, generic OTLP/HTTP receiver. Any agent
framework that speaks OpenTelemetry (Cowork OTel as a
sub-case, in-house agents, OpenInference-instrumented stacks)
can push to a per-source endpoint. `parserConfig`:
`{ allowedSourceTypeLabel? }`.
You can't block in-flight here, but you can:
1. Detect (anomaly rules, "user X did N actions outside business
hours").
2. Alert (Slack, PagerDuty, SIEM via OCSF).
3. **Trigger admin-API revoke**: call the platform's own admin
API (Anthropic, OpenAI, Microsoft Power Platform) to disable
the offending workspace key or session within seconds. Tier 3 is
"near-real-time stop", not "no stop".
The IngestionSource entity that holds the connection metadata is
shared across Tier 3, Tier 4, and the S3 audit drop case. See the
admin setup doc for the per-source forms.
## Tier 4: OpenTelemetry, SDK instrumentation
Customer-side agents that emit OpenInference, Traceloop, standard
OTel into LangWatch. This is the deepest visibility short of running
the runtime, per-turn detail, full trace context, prompt + response
in the trace, all the LangWatch eval/monitoring features attach
automatically.
Two important Tier 4 sub-cases:
* **Claude Cowork OTel export.** Anthropic ships Cowork with a
central OTLP-endpoint config, admin pastes the LangWatch URL +
bearer token in the workspace settings, every Cowork install in
the org streams to LangWatch. **No per-desktop config required.**
* **In-house agents using the LangWatch SDKs**: the existing
`langwatch` SDK setup; nothing governance-specific needed.
## Tier 5: Sandboxed runtime (Open Managed Agents)
For workloads the customer wants to run inside LangWatch's own
runtime: Open Managed Agents, Hermes, OpenClaw, or any custom
agent. Per gateway.md the recommended 2026 stack is **Firecracker
microVMs with snapshot-restore** for cloud, **Kata Containers, Google
Agent Sandbox on Kubernetes** for self-hosted. Egress policy via
Envoy sidecar, MCP allowlist enforced at the runtime layer.
Tier 5 is post-Phase-1 work; it's not in the current PR. Linked
here so the broader story is visible.
## Where this PR ships in the ladder
The current branch (PR #3524) implements Tier 1 end-to-end:
* ✅ **CLI** (`langwatch login --device`, `langwatch claude/codex/
cursor/gemini`, `langwatch whoami`, `langwatch me`,
`langwatch logout`, `langwatch init-shell`,
`langwatch request-increase`).
* ✅ **Personal Workspace** (Team + Project with `isPersonal` flag).
* ✅ **Personal Virtual Keys** (issued via the device-flow exchange).
* ✅ **Routing Policies** (Org/Team/Project hierarchy, provider-cred
validation).
* ✅ **`/me` + `/me/settings`** dashboards (web app, gated behind the
`release_ui_ai_governance_enabled` flag).
* ✅ **`/settings/governance`** admin oversight dashboard
(org-admin-only via `organization:manage` permission, plus the
governance flag). Wired to real `api.activityMonitor.{summary,
spendByUser, ingestionSourcesHealth, recentAnomalies}` tRPC
procedures; **no mocked fixtures**: empty data renders a setup
checklist (publish a RoutingPolicy, connect an IngestionSource),
data renders live cards + per-user table + source-health strip.
Read-mostly daily-use surface; admin authoring lives at the
`/settings/governance/*` sub-routes.
* ✅ **Budget enforcement** (cross-surface, gateway 402 ↔ CLI
Screen-8 box ↔ web `BudgetExceededBanner` all derive from
`GatewayBudgetService.check()`).
* ✅ **Helm `NOTES.txt` + post-install checklist** for self-hosted
bootstrap.
* ✅ **`IngestionSource` schema** (Postgres) + `trace_summaries.SourceType`
* `SourceId` columns (ClickHouse), the data model for Tier 3
* Tier 4. Seven `sourceType` values supported on the schema
side (see Tier 3 above).
* ✅ **`IngestionSourceService` + REST receivers + tRPC admin
router.** `POST /api/ingest/otel/:sourceId` (Tier 4 entry
point) and `POST /api/ingest/webhook/:sourceId` (Tier 3 push
destination) accept events authenticated by a per-source
`Bearer lw_is_*` secret, return 202, flip `lastEventAt` on the
source. `api.ingestionSources.{list,get,create,update,
rotateSecret,archive}` exposes the management surface to the
admin UI; raw secret is returned exactly once on create /
rotate, never echoed back afterwards.
* ✅ **`api.governance.setupState`**, the persona-routing
signal. Read-only summary of whether the org has personal VKs,
routing policies, ingestion sources, anomaly rules, or recent
activity. The boolean OR (`governanceActive`) drives whether
the `/governance` nav entry appears for org admins. Wires the
conservative no-auto-redirect promotion rule.
* ✅ **MainMenu Governance entry**: gated on the three-of-three
predicate: `release_ui_ai_governance_enabled` flag on AND
`organization:manage` permission AND
`api.governance.setupState.governanceActive === true`. Renders
a "GOVERN · Preview" section between LIBRARY and AI GATEWAY ·
Beta with an Eye-icon link to `/settings/governance`. Vast
majority of current LLMOps admins (no governance state) see no
nav change at all.
In flight on the same branch:
* 🚧 IngestionSource setup forms (per-source-type, in lane B):
wires to `api.ingestionSources.create`, surfaces the
one-time secret in a "paste this into upstream" modal.
* 🚧 Architecture deep-dive doc (the diagram + per-tier internals).
## Verified end-to-end (D2 foundation)
The IngestionSource auth path, the OTel receiver, and the
`lastEventAt` flip were dogfooded against `pnpm dev` on `:6660`.
The transcript IS the proof artifact (no UI to screenshot at this
layer, that's lane B's setup form, in flight).
```
$ pnpm tsx tmp/dogfood-ingestion.ts
✓ Planted IngestionSource
org: 123123123-NiG5AY
id: uotNPpD9TpASqQHTv6OXo
name: dogfood-otel-1777270498068
status: awaiting_first_event
ingestSecret: lw_is_kgOa878p…REDACTED
$ curl -sS -X POST \
-H 'Authorization: Bearer lw_is_kgOa878p…REDACTED' \
-d '{"resource_spans":[…]}' \
:6660/api/ingest/otel/uotNPpD9TpASqQHTv6OXo
{"accepted":true,"bytes":68}
HTTP 202
$ pnpm tsx tmp/dogfood-ingestion.ts --check
status: active ← flipped from awaiting_first_event
lastEventAt: 2026-04-27T06:15:03.361Z
```
Auth matrix verified in the same dogfood pass:
| Authorization header | Path source-id | Expected | Observed |
| ------------------------- | -------------- | ------------------------------------------ | ------------- |
| (none) | valid | 401 | 401 |
| `Bearer lw_is_unknown` | valid | 401 | 401 |
| `Bearer ` | wrong | 401 | 401 |
| `Bearer ` | matching | 202 | 202 |
| `Bearer ` (replay) | matching | 202 + `lastEventAt` advances monotonically | 202, advanced |
### Web UI full loop (Lane B dogfood)
The same backend, exercised end-to-end through the admin UI:
admin opens `/settings/governance/ingestion-sources`, fills the
composer, hits Save, a one-time-reveal modal exposes the
`lw_is_*` secret + OTLP URL, the admin copies, curls the
upstream, the list page polls and flips the source from
**Awaiting first event** → **Active** with "last event Ns ago".
The hero shot is the list page after the flip:
Full 6-step carousel for reviewers:
| # | Step | Shot |
| - | -------------------------------------------------------------- | ------------------------------------------- |
| 1 | `/settings/governance/ingestion-sources`, empty state | [open](https://i.img402.dev/wx4v7m7a8g.png) |
| 2 | Composer open, `sourceType` picker + dynamic per-type form | [open](https://i.img402.dev/ycld8afavo.png) |
| 3 | Composer filled, Generic OTel example | [open](https://i.img402.dev/yixqegurj3.png) |
| 4 | One-time secret reveal modal, `ingestSecret` + OTLP URL + Copy | [open](https://i.img402.dev/w4fl31ukrc.png) |
| 5 | List polled, flipped to **Active**, last event 13 s ago | [open](https://i.img402.dev/supafck5au.png) |
The dogfood walks the full Tier-3 push contract: admin issues
secret in UI → upstream curl with that secret → 202 ack →
`lastEventAt` flips → admin sees the flip on the next poll.
Secret is shown exactly once and never returned by the list/get
endpoints afterwards.
### CLI ingest debug (Lane A dogfood)
The same backend, exercised through the CLI without ever opening
a browser. After `langwatch login --device`, four read-only
governance debug helpers go live:
```text theme={null}
$ langwatch governance status
Governance setup state
· Personal VKs
· Routing policies
✓ Ingestion sources
✓ Anomaly rules
✓ Recent activity (30d)
Governance active: yes
$ langwatch ingest list
NAME TYPE STATUS LAST EVENT
Dogfood OTel · alexis lane otel_generic active 46m ago
$ langwatch ingest health JxQztBgN90QeX12-afJ0C
Dogfood OTel · alexis lane (JxQztBgN90QeX12-afJ0C)
Status: active
Events (24h): 1
Events (7d): 1
Events (30d): 1
Last event: 47m ago
$ langwatch ingest tail JxQztBgN90QeX12-afJ0C --limit 5
2026-04-27T07:11:18.963Z api.call chat_completion → claude-3-5-sonnet
```
Every command also accepts `--json` for machine output (exact
contract match with the equivalent tRPC procedure, web UI render).
`langwatch ingest tail --follow` polls every 3 s and dedupes by
`eventId`, so the CLI and the per-source detail page show the
same event stream byte-for-byte.
Auth for these REST adapters reuses the same Bearer
`access_token` the device flow persists in
`~/.langwatch/config.json`, no separate token. Unauthenticated
calls bounce with `Not logged in — run \`langwatch login --device\` first`;
unknown source IDs surface the underlying tRPC `NOT\_FOUND`as`Error: IngestionSource not found`. Both error paths are
asserted in `specs/ai-gateway/governance/cli-ingest-debug.feature\`
(12 scenarios).
### Admin oversight wired to real data (no mocks)
Once a source is `active` and events are flowing, the admin
oversight dashboard at `/settings/governance` queries
`gateway_activity_events` via the `api.activityMonitor.*` tRPC
procedures. Empty data renders a setup checklist; data present
renders live cards + per-user table + source-health strip.
| Surface | Real-data state | Shot |
| ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------- | ------------------------------------------- |
| `/settings/governance`, setup checklist + first real source visible in the health strip | live (`api.activityMonitor.{summary, spendByUser, ingestionSourcesHealth, recentAnomalies}`) | [open](https://i.img402.dev/hhghiq7gbb.png) |
| `/settings/governance/ingestion-sources`, list page after the iter-13 customer-name scrub | live (`api.ingestionSources.list`) | [open](https://i.img402.dev/kfvazcuczp.png) |
| `/settings/governance/anomaly-rules`, empty list (no rules yet) | live (`api.anomalyRules.list`) | [open](https://i.img402.dev/9fjdw9we08.png) |
| `/settings/governance/anomaly-rules`, composer (name + severity + ruleType + scope + JSONB threshold/destination configs) | live (`api.anomalyRules.create`) | [open](https://i.img402.dev/9v33n8zzrk.png) |
| `/settings/governance/anomaly-rules`, list after Create roundtrip; persisted rule grouped under Critical with the honest "rules persist now; evaluation + alerts arrive with the detection backend" banner | live (PG persistence; eval engine = Sergey Option C, deferred) | [open](https://i.img402.dev/15z3h6k32v.png) |
Tier 2, Tier 5 are deferred; the architecture doesn't preclude them
and the control-plane primitives already accommodate them.
## Feature-flag gates
Governance sits behind a single fence on each surface:
* **Web app:** the `release_ui_ai_governance_enabled` flag.
Enabled by default on self-hosted; on SaaS, PostHog release
conditions decide per organization.
Existing AI Gateway menu surface stays on the unrelated
`release_ui_ai_gateway_menu_enabled` flag, they ship
independently.
* **CLI:** always available once installed. Per-account governance
entitlement is enforced server-side, so the surface ships without
a client-side flag.
* **Backend:** the tRPC procedures, REST endpoints, gateway 402
payload, and Prisma schema are not flag-gated. Hiding only the
user-facing surfaces lets the long-lived branch merge cleanly
without exposing preview features.
## Where to next
* For a dev opting in to Tier 1 today:
[Personal IDE keys](/docs/ai-gateway/governance/personal-keys).
* For an admin onboarding a new org:
[Admin setup](/docs/ai-gateway/governance/admin-setup).
* For the routing-policy DSL that lets one VK serve multiple tools:
[Routing policies](/docs/ai-gateway/governance/routing-policies).
* For per-tool CLI config (env vars, dialect notes):
[Coding CLI Integrations](/docs/ai-gateway/cli/overview).
* For the architecture deep-dive once published:
`architecture.md` (in flight on this branch).