> ## Documentation Index
> Fetch the complete documentation index at: https://langwatch.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Ingestion sources

> Per-platform integration matrix for LangWatch governance receivers. Each source picks an OTLP wire shape (spans for span-shaped feeds, logs for flat audit feeds) and lands in the same unified trace store that powers the rest of LangWatch.

<Info>**Pairs with:** [AI Gateway → Observability](/docs/ai-gateway/observability). The gateway is itself an ingestion source (its spans land in the same trace store); IngestionSources extend that substrate to platforms whose API key path you don't own (Workato, Copilot Studio, OpenAI/Anthropic compliance APIs). Also pairs with [Audit log](/docs/ai-governance/audit-log): S3 audit log feeds (Workato, Copilot Studio, Anthropic Compliance) become OTel-shaped events through this same pipeline, so a single audit log query at `/settings/audit-log` covers in-platform mutations and third-party-tool activity uniformly.</Info>

<Note>**`otel_generic` + `thirty_days` retention are Apache 2.0 floor; multi-source + extended retention require Enterprise.** Self-hosted Apache 2.0 deployments can wire one OTel receiver at the default 30-day retention; the additional source types on this page (Workato, S3, Copilot Studio, OpenAI, Anthropic compliance APIs) and the `one_year`, `seven_years` retention classes ship under the Enterprise license. Non-enterprise organizations see an upgrade card on `/settings/governance/ingestion-sources`. See [Open-core licensing](/docs/ai-governance/overview#open-core-licensing) for the full split.</Note>

LangWatch governance ingest accepts events from upstream agent and audit-log platforms and lands them in the **same `recorded_spans` + `log_records` tables** that power the rest of LangWatch, distinguished by `langwatch.origin.kind = "ingestion_source"` origin metadata stamped at the receiver edge. Same trace viewer, same compliance + RBAC + retention machinery, surfaced in the [governance dashboard](/docs/ai-governance/control-plane), the per-source detail page, and the [`langwatch ingest tail`](/docs/ai-governance/cli-debug) CLI.

<Note>**Governance ingestion is for third-party AI platforms your org runs.** If you want to send your *own application's* LLM traces to LangWatch, use [`/api/otel/v1/traces`](/docs/integration/quick-start) with your project API key, that's the right home for traces from systems you own. See [Choosing the right OTel endpoint](/docs/observability/trace-vs-activity-ingestion) for the full comparison.</Note>

This section documents each supported `sourceType` honestly: what's wired up today, what's still envelope-only, and what's a configuration contract awaiting an adapter.

## State of each receiver: at a glance

| Source type                                                                       | Delivery                           | OTLP shape | Storage          | State                                                                         |
| --------------------------------------------------------------------------------- | ---------------------------------- | ---------- | ---------------- | ----------------------------------------------------------------------------- |
| [`otel_generic`](/docs/ai-gateway/governance/ingestion-sources/otel-generic)           | Push (HTTP/OTLP)                   | Spans      | `recorded_spans` | **Production-ready**                                                          |
| [`claude_cowork`](/docs/ai-gateway/governance/ingestion-sources/claude-cowork)         | Push (HTTP/OTLP)                   | Spans      | `recorded_spans` | **Receiver works**, Cowork-specific attribute mapping pending                 |
| [`workato`](/docs/ai-gateway/governance/ingestion-sources/workato)                     | Webhook → OTLP logs                | Logs       | `log_records`    | **Receiver works**, deeper audit-shape parser pending                         |
| [`s3_custom`](/docs/ai-gateway/governance/ingestion-sources/s3-custom)                 | S3 replay + callback webhook       | Logs       | `log_records`    | **Callback-mode works**, S3 puller + DSL parser pending                       |
| [`copilot_studio`](/docs/ai-gateway/governance/ingestion-sources/copilot-studio)       | Pull (Purview Audit API)           | Logs       | `log_records`    | **Setup-contract-only**: Azure AD app config persisted, puller worker pending |
| [`openai_compliance`](/docs/ai-gateway/governance/ingestion-sources/openai-compliance) | Pull (Enterprise Compliance JSONL) | Logs       | `log_records`    | **Setup-contract-only**: S3 + role config persisted, puller worker pending    |
| [`claude_compliance`](/docs/ai-gateway/governance/ingestion-sources/claude-compliance) | Pull (Anthropic Compliance API)    | Logs       | `log_records`    | **Setup-contract-only**: workspace API key persisted, puller worker pending   |

**Production-ready** = receiver accepts traffic, lands in unified store, web detail page + CLI tail render with full payload + cost, token deltas, drill-down opens the trace viewer (span-shape) or log-detail pane (log-shape).

**Receiver works** = endpoint accepts and ack's traffic, source flips to `active`, the unified store has the event with origin metadata stamped, but per-platform deeper attribute extraction beyond the default OTLP/JSON shape is pending. Drill-down still works, but column displays may show generic labels until the platform-specific adapter ships.

**Setup-contract-only** = the source can be created via the admin composer and persists the per-platform fields the eventual puller worker will use, but no traffic flows yet. Creating a source mints the credential + persists the config, but `langwatch ingest health` will show zeros indefinitely until the worker ships.

## Why two OTLP shapes (spans vs logs)

Different upstreams emit different event shapes:

* **Span-shaped sources** (`otel_generic`, `claude_cowork`) emit native OTLP spans with parent-child relationships, durations, and span kind. They benefit from drill-down in the trace viewer (showing the multi-step agent activity tree).
* **Flat-event sources** (`workato`, `s3_custom`, `copilot_studio`, `openai_compliance`, `claude_compliance`) emit one event = one row, no span tree. Forcing them into the span shape requires synthetic `traceId`, `spanId`, duration that carry no information. They land naturally as OTLP `log_records` and drill into the log detail pane.

**One internal pipeline either way**: both shapes pass through the same hardened OTLP parser (`parseOtlpBody.ts`) and the same trace pipeline downstream. Both are queryable from the same governance dashboard, both pull into the OCSF v1.1 read API, both honour the per-origin retention class.

## How a source becomes "active"

Independent of source type, the lifecycle is:

1. Admin opens `/settings/governance/ingestion-sources`, picks a source type, fills the per-type config form (including the [retention class](/docs/ai-gateway/governance/retention)), clicks Create.
2. Backend mints a `lw_is_<base64url>` ingest secret, hashes it, persists the source with status `awaiting_first_event`. The secret is shown **once** in a one-time-reveal modal, it's never returned by the list/get endpoints again.
3. Admin pastes the secret + per-source endpoint into the upstream platform (OTLP exporter URL, webhook destination, Azure app secret, S3 bucket policy, etc.).
4. First event arrives at the receiver → receiver lazy-ensures the hidden Governance Project for the org (if not already), stamps `langwatch.origin.*` + `langwatch.governance.retention_class` attributes, hands off to the existing trace pipeline. Source status flips to `active` + `lastEventAt` updates.
5. Web detail page polls every 10s, CLI tail polls every 3s in `--follow` mode, both see the flip + the new event simultaneously.

For source types where the puller worker is not yet wired (the three "setup-contract-only" rows above), step 4 won't fire; the source stays at `awaiting_first_event` until the adapter ships.

## Auth contract: same shape across every receiver

All push-mode + webhook receivers use `Authorization: Bearer lw_is_<secret>` (the same shape the gateway uses for its `vk-lw-` virtual keys, just a different prefix). Mismatched `:sourceId` path vs. resolved-secret source ID returns `401 unauthorized` without leaking which one exists. The 24-hour rotation grace window is documented in the IngestionSource lifecycle architecture.

## Verification: same loop for every active source

Once a source is active, verify with the [governance CLI debug helpers](/docs/ai-gateway/governance/cli-debug):

```bash theme={null}
# What does the org see?
langwatch ingest list

# How healthy is one source?
langwatch ingest health <sourceId>

# What events landed in real time?
langwatch ingest tail <sourceId> --follow
```

The CLI hits the same backend the web detail page hits, byte-for-byte (`--json` mode is contract-stable against `api.governance.eventsForSource` and `api.ingestionSources.healthMetrics`).

## Origin metadata: what every event carries

Every governance-ingested span, log\_record is stamped at the receiver edge with reserved attribute namespaces (rejected if found in user-supplied OTLP):

| Attribute                                    | Purpose                                                                   |
| -------------------------------------------- | ------------------------------------------------------------------------- |
| `langwatch.origin.kind`                      | Always `"ingestion_source"` for governance traffic, discriminator         |
| `langwatch.ingestion_source.id`              | Source identity (matches the `lw_is_*` Bearer's resolved IngestionSource) |
| `langwatch.ingestion_source.organization_id` | Org tenancy for cross-source roll-ups                                     |
| `langwatch.ingestion_source.source_type`     | Source type (`otel_generic`, `workato`, etc.) for filtering               |
| `langwatch.governance.retention_class`       | `thirty_days`, `one_year`, `seven_years`, drives ClickHouse TTL           |

These appear as read-only system metadata in the trace viewer, log detail pane; users cannot supply or edit them.

## Adapter roadmap

The order in which deeper per-platform adapters land depends on customer pull. The current priority order:

1. **Cowork attribute extraction**: Cowork pushes OTLP today; receiver accepts but spans aren't mapped to Cowork's `tool_use` taxonomy yet. Smallest gap to close.
2. **Workato audit-shape parser**: Workato has a stable JSON envelope for job-completed events; the default mapper retains the raw envelope; deeper Actor, Action, Target extraction is the follow-up.
3. **Microsoft Copilot Studio (Purview Audit) puller**: needs an Azure AD app + cron worker; the config is persisted but the worker isn't.
4. **OpenAI, Anthropic Enterprise Compliance pullers**: S3 + Anthropic compliance API respectively; config persisted, pullers pending.
5. **`s3_custom` DSL parser**, generic-shape parser for homegrown audit logs; the customer-facing DSL design is in flight.

If you need an unwired adapter, file an issue tagged `governance/ingestion/adapter-roadmap` and we'll sequence accordingly.

## Cross-references

* [Trace vs governance ingestion](/docs/observability/trace-vs-activity-ingestion): picking the right OTel URL
* [Compliance architecture](/docs/ai-gateway/governance/compliance-architecture): how the unified substrate underwrites SOC 2, ISO 27001, EU AI Act, GDPR, HIPAA-most-uses
* [Per-origin retention](/docs/ai-gateway/governance/retention): `thirty_days`, `one_year`, `seven_years` classes
* [OCSF, SIEM export](/docs/ai-gateway/governance/ocsf-export): pulling governance events into Splunk, Datadog Security, etc.
* [Governance CLI debug](/docs/ai-gateway/governance/cli-debug): `langwatch ingest list/health/tail` commands