# Open source LLM observability in 2026: what runs in your cluster and what calls home

Open source LLM observability you can self-host: what a LangWatch install runs, the one daily report it sends home and how to switch it off, air-gapped installs with offline licenses, and how it compares with Langfuse, Arize Phoenix and LangSmith.

*By Rogerio Chaves · October 24, 2026*

Canonical: https://langwatch.ai/blog/self-hosted-llm-observability

![Open source LLM observability in 2026: what runs in your cluster and what calls home](https://langwatch.ai/blog/self-hosted-llm-observability-cover.webp)

**Short answer:** Yes, you can run open source LLM observability entirely inside your own network. LangWatch, Langfuse and Arize Phoenix all self-host for free. Before you pick one, check three things: what the install runs, what it sends home, and which enterprise controls need a paid license.

We build [LangWatch](https://langwatch.ai), an open-source platform for tracing, evaluating and testing LLM apps and agents, licensed under Apache 2.0. This post uses our self-hosted install as the worked example, because it's the one I can describe down to the last outbound connection.

Platform teams in banks, healthcare and the public sector tend to ask the same questions about self-hosting, and in roughly the same order. A healthcare group's procurement team wanted an architecture diagram before anyone was allowed to test. Their monitoring lead's first question was what the install calls home.

## Can you self-host open source LLM observability?

Yes. Open source LLM observability tools publish their images and Helm charts, so you can run the whole stack in your own cluster with your own databases. The difference between tools is how much of the product comes with the free install, and what still talks to the vendor.

Self-hosted LangWatch runs the same software as LangWatch Cloud. There's no separate community build to outgrow. Without a license it has no caps: unlimited members, teams, projects, traces, evaluations, simulations and experiments. An Enterprise license switches on SSO, SCIM, custom roles, audit logs and longer retention in the deployment you already run, so going enterprise later doesn't mean reinstalling ([editions and licensing](https://langwatch.ai/docs/self-hosting/licensing)).

You can also split it. In the hybrid setup we run the app, the workers and the evaluation services, and you keep ClickHouse and S3 in your own VPC, so the trace data never leaves your network.

## What are the best open source LLM observability tools?

The main self-hostable options are LangWatch (Apache 2.0), Langfuse (MIT core), Arize Phoenix (Elastic License 2.0) and LangSmith, which you can self-host but isn't open source. If you want tracing, evals, agent testing and an AI gateway in one install, that's LangWatch.

| Tool | License | What one self-hosted install includes | SSO, RBAC and audit logs | Calls home, air-gapped |
|---|---|---|---|---|
| LangWatch | Apache 2.0; enterprise code sits in a separate `enterprise/` directory | Tracing, evaluations, agent simulations, prompt management, the AI Gateway and the in-product assistant, with ClickHouse, PostgreSQL, Redis and S3, in one Helm chart | Built-in roles on every install; SSO, SCIM, custom roles and audit logs with an Enterprise license | One daily report with counts only, plus a license sync when the license names hosted services. One switch stops every connection, and a signed license works offline |
| Langfuse | MIT core; some features need a license key | Web and worker containers with PostgreSQL, ClickHouse, Redis and S3; an LLM API for the playground and evals | SSO included; project-level RBAC roles, audit logs and SCIM need the license key | Aggregated usage telemetry, which you can switch off on the open source edition. On Enterprise it stays on for license compliance. Internet access is optional |
| Arize Phoenix | Elastic License 2.0, free to self-host | One container (UI and trace collector) on SQLite or PostgreSQL; one instance is one tenant | OAuth2, LDAP and RBAC included, no feature gates | UI web analytics, off with one variable. A dedicated setting turns off every public-internet request for air-gapped installs |
| LangSmith | Proprietary | Frontend, backend, queue and playground services with ClickHouse, PostgreSQL, Redis and blob storage | Self-hosting itself is an add-on to the Enterprise plan | Trace data stays in your environment |

OpenLIT, Opik and Helicone are also Apache 2.0 and self-hostable. OpenLIT is the lightest of them: one container with a built-in OTLP receiver, plus ClickHouse. For hosted tools as well, see our [LLM observability tools list](https://langwatch.ai/blog/top-10-llm-observability-tools-complete-guide-for-2026), and [what agent observability is](https://langwatch.ai/blog/what-is-agent-observability) if your team is still deciding what to trace.

Langfuse and Phoenix give you SSO without a license, and that's a fair point in their favor. Where LangWatch pulls ahead is everything that comes after tracing, in the same install. Evaluations, agent testing with simulated users, prompt management and the AI Gateway all run in your cluster and write to the same ClickHouse, and none of them carries a usage cap. Every outbound connection is written down in a table with the switch that stops it, and the switch also works on a licensed install, so a paid install can still run with no connection to us at all.

## What a self-hosted install runs, component by component

A self-hosted LangWatch is one Helm chart. It runs the app, the workers, the AI Gateway, the Langy agent, PostgreSQL, Redis, ClickHouse with S3 storage, and the NLP and evaluation services. Only the app needs an ingress; everything else stays on cluster-internal services.

To try it, add the chart repo, pull the chart and install it:

```bash
helm repo add langwatch https://langwatch.github.io/langwatch
helm pull langwatch/langwatch --untar
helm install langwatch ./langwatch --namespace langwatch --create-namespace --set autogen.enabled=true -f langwatch/examples/overlays/size-dev.yaml --wait --timeout 10m
```

The dev profile fits a small cluster at about 2.8 CPU and 6.9 Gi of memory requests, and it generates its own secrets, so you can try it today. For production you replace those secrets with your own, because a reinstall regenerates them.

| Component | What it does | Exposed outside the cluster |
|---|---|---|
| App | Web UI, REST API, OpenTelemetry (OTLP) ingestion, sign-in | Yes, the only required ingress |
| Workers | The processing pipeline: cost enrichment, PII redaction, evaluations, alerts | No |
| LangEvals and NLP | Built-in evaluators and guardrails; workflow and playground model calls | No |
| AI Gateway | OpenAI-compatible proxy with virtual keys, budgets, routing and guardrails | Optional, when apps outside the cluster call it |
| Langy agent | The in-product assistant's workers | No |
| PostgreSQL | Users, teams, projects, settings, prompt versions | No |
| ClickHouse with S3 | Traces, spans, evaluations, experiments; older data moves to S3 | No |
| Redis | Job queue and cache | No |

Your agents send traces over OpenTelemetry, so anything already instrumented with OTel works without our SDK. PII redaction runs on the workers before a trace is stored, and it's on by default. The strict level runs Presidio inside the cluster.

It scales the boring way. A trial fits a small cluster, workers are stateless and you add replicas when the queue grows, and production ClickHouse runs replicated on three or more nodes. If you already run managed PostgreSQL, Redis or ClickHouse, the chart has overlays to point at them instead of the bundled ones. The [architecture page](https://langwatch.ai/docs/self-hosting/infrastructure/architecture) has the diagram procurement asks for, ports and data flow included.

## What leaves your network, and what never does

A self-hosted LangWatch sends one usage report a day with counts and metadata. A licensed install with hosted services also syncs its license daily. Neither carries a trace, prompt, dataset row, evaluation input or output, name, email address, IP address or key, and each has a switch.

The [data and telemetry](https://langwatch.ai/docs/self-hosting/data-and-telemetry) page lists every host the install can reach, one row each, with the component that calls it, when, what it carries and how to stop it. The short version:

- **The usage report.** Once a day: counts, timestamps, the release, and the company as email domains with counts, for example `acme.com: 14`. `DISABLE_USAGE_STATS=true` stops it. You can read the exact payload your install would send on **Settings, Checkup**, or with `langwatch doctor`.
- **The license sync.** Only for licenses that name a hosted service: a hash of the license key, the instance id, the release and seat counts. An offline license never sends it.
- **Everything at once.** `LANGWATCH_CONNECT_DISABLED=true` stops every connection to LangWatch, the report included. With it set, the install builds no client and opens no connection to us for any reason, which helps when an audit has to prove absence rather than trust a config review.

Two hosted services do carry content, and only when your license names them and no admin switched them off. Instant Evals sends the text being judged and your questions to our gateway. Managed models send the full model request when someone calls a `langwatch/` model through the AI Gateway. An admin can switch either off on **Settings, Connect**, and the kill switch above stops both.

Calls that libraries inside the images would make on their own are off by default: Prisma's version check, RAGAS analytics, tokenizer and price list downloads, the Cloudflare tunnel for voice. The images ship the tokenizer files and model prices they need.

Then there's the traffic you choose. Model calls go to the providers you configure, with your keys. Webhooks, Slack, email and SSO go where you point them. If some content shouldn't be stored at all, the [data privacy policy](https://langwatch.ai/docs/platform/data-privacy) drops inputs, outputs, system prompts or tool calls at ingestion and keeps cost, tokens and latency, per organization, team or project.

## Air-gapped installs and offline licenses

To run LangWatch with no internet access, mirror the images into your private registry, install the Helm chart from there, and use a signed license key, which the install verifies locally. Traces, evaluations and agent simulations then stay inside your network.

A distributor selling to banks in North Africa summed up what their clients need in one line on a call with us: "Self-hosted means that I can have my own cloud." Those banks run isolated AI stacks with no access to frontier-model APIs, so the install has to work with the internet cable pulled.

The steps:

1. **Mirror the images.** The app, NLP and LangEvals images, then the gateway, Langy, ClickHouse, PostgreSQL and Redis images, all pointed at your registry in the chart values.
2. **Use the signed license key.** A license comes in two forms. An activation code needs one outbound call to redeem. A signed key, a long string or a `.langwatch-license` file, is verified locally and works offline.
3. **Turn off the outbound paths.** Set `LANGWATCH_CONNECT_DISABLED=true` and switch off usage stats. With no hosted services named in the license, Instant Evals and managed models never run.
4. **Bring a judge model you can reach.** LLM-as-a-judge evaluators and simulated users need a model. Point them at one inside the network, through a custom endpoint or a provider your cloud runs privately. Evaluators that use no LLM, like PII detection, run in the cluster as they are.

Supply chain checks fit the same setup. Every image is signed with Sigstore cosign and comes with a CycloneDX SBOM, so you can verify an image before it enters your registry.

## One install for every team or one per agent?

One install for everyone is what we recommend. LangWatch is multi-tenant: organizations, teams and projects, with every query scoped to its tenant. Run a separate install only when a regulation or policy needs hard separation, and even then, it's another Helm release on another cluster.

A Nordic bank's platform team had a separate tracing deployment for each agent, because of anti-money-laundering compliance. The question I get from platform teams like theirs is whether LangWatch needs the same, and whether dev, test and prod each need their own instance.

For load, one install takes all the teams, and you scale it by adding workers and ClickHouse nodes. Separation inside one install comes from three places:

1. **Projects and teams.** Each project has its own traces, keys and settings. Project keys are scoped to one project, and every ClickHouse query filters by tenant first.
2. **Roles.** The built-in Admin, Member and Viewer roles work on every install. Custom roles, groups and role bindings come with the Enterprise license.
3. **Privacy rules per scope.** A rule can restrict who reads a project's content or drop it at ingestion, so the compliance team can see one agent's conversations and the rest of the bank sees cost and latency.

When the rules need more than that, say a regulator wants one agent's data on separate infrastructure, install the chart again on that cluster. It's the same software and the same Helm command, so the second install costs you the cluster it runs on. One signed license key in the configuration covers every organization inside that one install.

## Checklist for the security review

Bring the architecture page, the outbound connection table, the license split and the hardening defaults. Reviewers ask about those four first: what runs, what leaves, what's gated, and how the pods are locked down.

For the commercial side, DPAs, ISO 27001 reports and help with InfoSec questionnaires, see the [enterprise page](https://langwatch.ai/enterprise). Before the review:

- **Architecture.** Components, ports and data flow from the architecture page. Only the app is exposed, on 443.
- **Outbound connections.** The data and telemetry table, with `LANGWATCH_CONNECT_DISABLED` for no egress to LangWatch at all.
- **Identity.** SSO through Okta, Azure AD, Auth0, Google, GitHub or GitLab, and SCIM provisioning, both with the Enterprise license.
- **Access and audit.** Built-in roles, custom roles, and the audit log of every change made through settings, the API and the AI Gateway.
- **Data handling.** PII and secrets redaction on by default, content drop or restricted reading per scope, 49 days of retention by default and longer with a license.
- **Encryption.** Provider keys, SSO credentials and project secrets encrypted with AES-256-GCM in PostgreSQL. TLS at the ingress and to the databases.
- **Pods.** Non-root, read-only root filesystem, all capabilities dropped, no service account token. A `strict-admission` overlay covers clusters that enforce the restricted Pod Security standard.
- **Images.** Signed, with SBOMs attached, verifiable with `cosign verify` at pull time.

## Frequently asked questions

### What is the best self-hosted LLM gateway for enterprises?

One that runs inside your own perimeter and logs every call next to your traces. The LangWatch AI Gateway ships in the same Helm chart as the rest of LangWatch: an OpenAI-compatible endpoint with virtual keys, budgets, routing and fallbacks across providers, guardrails and prompt caching. It calls the model providers you configure with your own keys, and one chart value turns it off.

### Is Langfuse open source?

Yes. The Langfuse core is MIT licensed and runs the same infrastructure as Langfuse Cloud: a web and a worker container with PostgreSQL, ClickHouse, Redis and S3. A license key adds project-level RBAC roles, audit logs, data retention policies, SCIM and server-side data masking. Telemetry can be switched off on the open source edition, and on the Enterprise edition it stays on for license compliance.

### Does self-hosted LangWatch send data back to LangWatch?

It sends one usage report a day with counts and metadata, and a licensed install with hosted services also syncs its license once a day. Neither carries a trace, a prompt, a dataset row, a name, an email address or a key. DISABLE_USAGE_STATS=true stops the report, and LANGWATCH_CONNECT_DISABLED=true stops every connection to LangWatch.

### Can I run LLM observability in an air-gapped environment?

Yes. With LangWatch you mirror the images into your private registry, install the Helm chart in your own cluster, and use a signed license key, which is verified locally and works offline. Set LANGWATCH_CONNECT_DISABLED=true so nothing tries to reach the internet. LLM-as-a-judge evaluators need a model you can reach inside the network.

### Do I need a license to self-host LangWatch?

No. Self-hosted LangWatch is Apache 2.0 and uncapped without a license: unlimited members, projects, traces, evaluations, simulations and the AI Gateway, commercial use included. An Enterprise license adds SSO, SCIM, custom roles, audit logs, longer retention and a support contract, all in the same deployment, so adding one needs no reinstall.