> ## Documentation Index > Fetch the complete documentation index at: https://langwatch.ai/docs/llms.txt > Use this file to discover all available pages before exploring further. # LangWatch CLI > The `langwatch` CLI is a single tool that drives LangWatch from the terminal, for you and for your coding assistant. Instrument code, version prompts, run scenarios, inspect traces, query analytics, and more. The CLI is designed to be driven by a coding assistant, and it reaches everything in the LangWatch app except cache rules, which you manage over [REST](/docs/ai-gateway/api/management#cache-rules) or in the dashboard. Every subcommand supports `--help`, and every command accepts the output-contract flags (`-o json`, `--json `, `--jq `; see [Agent usage](#agent-usage)). Run `langwatch --help` to see the full command tree. ## Install The CLI needs **Node.js 18 or newer** (the `npm` command ships with it). If you don't already have it, get it from [nodejs.org](https://nodejs.org). Then install globally: ```bash theme={null} npm install -g langwatch ``` Or run any command with `npx`, no install required: ```bash theme={null} npx langwatch --help ``` `pnpm install -g langwatch` and `yarn global add langwatch` work the same way; the package is identical across all three package managers. ### Standalone binary (no Node.js required) Each release also attaches a self-contained binary for Linux, macOS, and Windows. Use it in containers, CI images, and anywhere you'd rather not install Node.js. Download the one for your platform from the [releases page](https://github.com/langwatch/langwatch/releases), then: ```bash theme={null} # Verify the download against the release's SHA256SUMS, then install it sha256sum --check --ignore-missing SHA256SUMS chmod +x langwatch-- sudo mv langwatch-- /usr/local/bin/langwatch ``` Every binary is published with a signed [build provenance attestation](https://docs.github.com/en/actions/security-guides/using-artifact-attestations-to-establish-provenance-for-builds), so you can confirm it really was built by LangWatch's release workflow from this repository: ```bash theme={null} gh attestation verify langwatch-- --repo langwatch/langwatch ``` #### macOS: clearing Gatekeeper The macOS binaries are **not yet codesigned or notarized**. Gatekeeper will refuse to run them with *"langwatch cannot be opened because the developer cannot be verified"*. macOS attaches a `com.apple.quarantine` attribute to anything downloaded from the internet. After verifying the checksum above, remove it: ```bash theme={null} xattr -d com.apple.quarantine langwatch--darwin-arm64 ``` Then run the binary as normal. (If `xattr` reports *"No such xattr"*, the attribute was never set, so there is nothing to do.) Only clear quarantine on a binary whose checksum you have verified against the release's `SHA256SUMS`. Signing and notarization are tracked as follow-up work; until then, `npm install -g langwatch` avoids Gatekeeper entirely and is the smoother path on macOS. ## Authenticate `langwatch login` is interactive by default, it asks where (cloud vs self-hosted) and how (AI tools vs project SDK), opens your browser to approve, and the credential flows back to the CLI automatically. **No copy-paste of keys**: ```bash theme={null} langwatch login ``` You'll see two questions: 1. **Where do you want to log in?**: LangWatch Cloud (`app.langwatch.ai`) or a self-hosted instance (custom URL). 2. **How do you want to use it?**: three options: * **AI tools, agentic flows**: `claude`, `codex`, `cursor`, `gemini`, `opencode`. Mints an OAuth-style device session in `~/.langwatch/config.json` (user-scoped) so `langwatch claude` etc. wrap any tool through your gateway. * **Project, SDK API key**: for `langwatch sync`, `langwatch eval`, and SDK auto-instrumentation. Writes the API key of the project you pick into `$CWD/.env` (project-scoped). * **Both**: runs both flows in sequence. The browser shows an "Approve" or "Send API key" button (depending on the chosen mode); after you click, the credential is delivered back to the CLI over the same RFC 8628 device-code poll. **You never copy-paste a key.** ### Storage discipline (where credentials land) | Scope | Path | What lives there | | ----------- | ---------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Project | `$CWD/.env` | `LANGWATCH_API_KEY`, used by SDK consumers + `langwatch sync`, `eval`, `prompt` | | User-global | `~/.langwatch/config.json` (mode `0600`) | Device session (`access_token` + `refresh_token`), control-plane URL, gateway URL, default org, used by `langwatch claude/codex/cursor/gemini/opencode`, `langwatch whoami`, `langwatch request-increase` | The two stores serve different audiences and never leak into each other. Logging out of one (`langwatch logout` clears `~/.langwatch/config.json`) doesn't touch the other, the project API key in `.env` stays put. ### Self-hosted The CLI picks up your self-hosted endpoint from any of these (highest priority first): | Priority | Source | Use case | | -------- | -------------------------------------------- | ----------------------------------------- | | 1 | `--endpoint ` flag on `langwatch login` | one-shot login at a different host | | 2 | `LANGWATCH_ENDPOINT` env var | CI, scripts | | 3 | `~/.langwatch/config.json:control_plane_url` | persisted from prior login (daily driver) | | 4 | `https://app.langwatch.ai` | built-in cloud default | The simplest path for self-hosted users is the interactive prompt, pick "Self-hosted instance", enter your URL, and the CLI saves it for future invocations: ```bash theme={null} langwatch login # → Where do you want to log in? ▸ Self-hosted instance # → URL: https://langwatch.acme.internal ``` Or, equivalently, one-shot with the flag: ```bash theme={null} langwatch login --endpoint https://langwatch.acme.internal ``` To inspect or change persisted settings without re-logging in: ```bash theme={null} langwatch config list # show resolved values + sources langwatch config get endpoint langwatch config set endpoint https://langwatch.acme.internal # writes ~/.langwatch/config.json ``` ### Non-interactive escape hatches (for CI, agents) When you're driving the CLI from automation and already have a credential, skip the prompts: | Flag | Use case | Where it lands | | ---------------------------------- | ---------------------------------------------------------------------------------------------------- | -------------------------- | | `langwatch login --device` | AI tools mode, skip Q1/Q2 prompts | `~/.langwatch/config.json` | | `langwatch login --project` | Project SDK mode via browser, skip Q1/Q2 prompts; mints a fresh project key | `$CWD/.env` | | `langwatch login --api-key ` | CI pipeline that has `LANGWATCH_API_KEY` injected from secrets | `$CWD/.env` | | `langwatch login --token ` | Agent harness with a pre-minted device session token (issued via dashboard "Personal Access Tokens") | `~/.langwatch/config.json` | | `langwatch login --endpoint ` | combine with any of the above to pin a self-hosted instance | persisted to config | The interactive `langwatch login` always shows these flags in a banner above the prompts, so a fake-TTY agent (`Claude Code`, certain Gemini CLI sandboxes) can detect the prompt and re-invoke with the right flag instead of getting stuck. When stdin is not a TTY (genuine CI or an agent's piped stdin), `langwatch login` with no flags defaults to **project login** (the same as `--project`): it mints a project key into `$CWD/.env`, which is what the SDK, `langwatch eval`, and `langwatch prompt` expect. AI-tools login stays explicit behind `--device`. ### Letting an agent do it A coding assistant driving `langwatch` will see the always-on banner naming `--device`, `--project`, `--api-key`, `--token`, `--endpoint` whenever the interactive prompt fires. If the assistant's harness reports as a TTY but can't actually answer prompts, the banner gives it everything it needs to re-invoke: ```bash theme={null} langwatch login --device # AI tools mode (recommended for assistants) langwatch login --project # project SDK key into .env via the browser langwatch login --api-key sk-lw-... # if a project key was injected from secrets langwatch login --token lwc_... # if a device session token was pre-minted langwatch login --endpoint https://... # combine with any of the above for self-hosted ``` ## Fetch documentation `langwatch docs` returns any LangWatch documentation page as plain Markdown, ideal for feeding into an agent's context before it writes code. ```bash theme={null} langwatch docs # Documentation index langwatch docs integration/python/guide # Python integration guide langwatch docs integration/typescript/guide # TypeScript integration guide langwatch docs prompt-management/cli # Prompts CLI reference langwatch docs integration/python/langgraph # Framework-specific guides ``` Scenario framework docs live under a separate namespace: ```bash theme={null} langwatch scenario-docs # Scenario index langwatch scenario-docs advanced/red-teaming ``` Both commands accept full URLs too, and any missing `.md` extension is appended automatically. If you're inside an assistant with no shell (for example, a chat-only environment), the same content is available over plain HTTP, append `.md` to any documentation path, e.g. `https://langwatch.ai/docs/integration/python/guide.md`. Indexes: [docs](https://langwatch.ai/docs/llms.txt), [scenarios](https://langwatch.ai/scenario/llms.txt). ## Version prompts The Prompts CLI turns your prompts into tracked files alongside your code, with lock files, tagging, and sync to the LangWatch platform. ```bash theme={null} langwatch prompt init # scaffold prompts.json + prompts/ folder langwatch prompt create my-agent # create a new local prompt langwatch prompt sync # push local changes, pull remote updates langwatch prompt list # see all prompts in the project ``` In your application code, fetch the latest version at runtime: ```python Python theme={null} import langwatch prompt = langwatch.prompts.get("my-agent") ``` ```typescript TypeScript theme={null} import { LangWatch } from "langwatch"; const langwatch = new LangWatch(); const prompt = await langwatch.prompts.get("my-agent"); ``` ### Tag versions for deployment Three built-in tags are available: `latest` (auto-assigned), `production`, and `staging`. Assign a tag to the current version: ```bash theme={null} langwatch prompt tag assign my-agent production ``` Then fetch by tag at runtime: ```python Python theme={null} prompt = langwatch.prompts.get("my-agent", tag="production") ``` ```typescript TypeScript theme={null} const prompt = await langwatch.prompts.get("my-agent", { tag: "production" }); ``` For canary or blue/green deployments, create custom tags with `langwatch prompt tag create`. For the full Prompts CLI reference, see the [Prompt Management CLI guide](/docs/prompt-management/cli). ## Run scenario tests Scenarios are the LangWatch equivalent of end-to-end tests for agents: a user simulator chats with your agent, an LLM judge scores the conversation against criteria you define, and everything is recorded for later inspection. ```bash theme={null} langwatch scenario list # see all scenarios langwatch scenario create "refund flow" \ --description "User asks for a refund on a recent order" \ --criteria "Agent verifies identity" \ --criteria "Agent processes the refund" langwatch scenario run --target ``` Group related scenarios into a **suite** (a run plan) for CI or scheduled runs: ```bash theme={null} langwatch suite list langwatch suite run # kicks off every scenario × target ``` ### Pass values into a run Every run command accepts a repeatable `--param key=value`. A parameter is a constant value that the whole run shares: a fixture id, a tenant, a plan, a region. ```bash theme={null} langwatch suite run --param account_tier=platinum --param region=eu-central langwatch scenario run --target http: --param account_tier=platinum langwatch experiment run --param model=gpt-5-mini langwatch workflow run --input '{"question":"where is my order?"}' --param region=eu-central ``` The value type follows the text you type. `true` and `false` are booleans. A value that parses as a number and prints back the same is a number. All other values stay text, so `007` and `1.50` stay text. When you repeat a name, the last value wins. Each command uses the values differently: * `suite run` and `scenario run` supply the parameters that the scenarios declare. A name that no scenario in the run declares is rejected before anything is scheduled. See [Scenario run parameters](/docs/agent-simulations/scenario-parameters). * `experiment run` merges the values into every dataset row. See [Running experiments in CI/CD](/docs/evaluations/experiments/ci-cd#choosing-what-to-evaluate). * `workflow run` sends the values as entry inputs. They are merged over `--input`, so a `--param` pair wins when both name the same key. ### Inspect simulation runs Every scenario execution produces a **simulation run** you can inspect after the fact, full conversation, judge verdict, reasoning, met/unmet criteria, cost, and duration. ```bash theme={null} langwatch simulation-run list # recent runs, with relative timestamps langwatch simulation-run list --status FAILED # only failures langwatch simulation-run list --name "refund flow" # filter by name substring langwatch simulation-run get # full details for one run langwatch simulation-run get --full # don't truncate long messages ``` The `get` command renders assistant thinking blocks and tool calls as readable plain text, no raw JSON dumps in your terminal. Use `--format json` on either command for structured output. For the full scenario testing guide, see the [Scenarios documentation](/docs/agent-simulations/introduction). ## Inspect traces Traces capture every LLM call your agent makes, prompts, responses, latency, cost, errors. Search and drill into them from the terminal: ```bash theme={null} langwatch trace search --limit 10 # most recent traces langwatch trace search --query "error" # full-text search langwatch trace get # one trace in detail langwatch trace export --output traces.jsonl # stream traces for offline analysis ``` Agents use this to debug their own instrumentation: after running your code once, ask the assistant to run `langwatch trace search --limit 5` and verify traces are flowing. If nothing appears, the instrumentation is wrong, no need to read logs. ## Query analytics Analytics aggregate your traces into performance and cost metrics without leaving the terminal: ```bash theme={null} langwatch analytics --help # available metrics and dimensions ``` Use it to answer questions like "what's my P95 latency this week", "how much did each agent cost last month", or "which prompts produce the most errors". The underlying data is the same as the LangWatch dashboard. ## Manage platform resources Every LangWatch resource follows the same consistent subcommand shape: ```bash theme={null} langwatch list langwatch get langwatch create [options] langwatch update [options] langwatch delete ``` Available resources include: * **`evaluator`**, create and version evaluators (answer correctness, faithfulness, custom LLM judges) * **`monitor`**, online evaluations that score production traces automatically * **`dataset`**, evaluation datasets (upload CSV, download, manage columns) * **`agent`**, agent definitions used by scenarios and monitors * **`dashboard`** and **`graph`**, custom analytics dashboards * **`trigger`**, automations (alerts, webhooks, dataset-append on failure) * **`secret`**, encrypted environment variables for scheduled agent runs * **`workflow`**, reusable workflows built in the UI * **`model-provider`**, configure OpenAI, Anthropic, Azure, or Bedrock for your project * **`annotation`**, attach labels to traces for supervised fine-tuning data Run `langwatch --help` on any of these for subcommand-level options, and `--format json` to get structured output for scripting. ## Organization management These commands manage org-wide resources and require an API key with organization-level permissions. The organization, members, invites, groups, roles, role bindings and SCIM token commands are available on Enterprise plans. Without one they exit with `enterprise_plan_required` and tell you what to do about it. Projects, teams, API keys and the self-hosted organizations command are not gated. ### Projects ```bash theme={null} # List all projects in your organization langwatch projects list langwatch projects list --format json # Create a project (returns a one-time service API key) langwatch projects create \ --name "My Agent Project" \ --language python \ --framework langchain \ --new-team-name "Engineering" # Get project details langwatch projects get proj_01HZX9K3MN... # Update project metadata langwatch projects update proj_01HZX9K3MN... --name "Renamed" # Archive a project (soft-delete) langwatch projects delete proj_01HZX9K3MN... ``` When creating a project, the CLI displays the service API key exactly once. Save it immediately. It cannot be retrieved later. ### API keys ```bash theme={null} # List all API keys langwatch api-keys list langwatch api-keys list --format json # Create a service key (org-wide) langwatch api-keys create --name "CI Pipeline" # Create a service key scoped to specific projects langwatch api-keys create \ --name "Staging Key" \ --project-id proj_01HZX... \ --project-id proj_02ABC... # Read one key and the access it carries langwatch api-keys get key_01HZX... # Rename it, or replace what it reaches langwatch api-keys update key_01HZX... --name "Staging (read only)" # Revoke a key (cannot be undone) langwatch api-keys revoke key_01HZX... ``` Both `create` and `update` take the access flags: `--binding role:scopeType:scopeId` says what the key reaches and where, `--permission resource:action` narrows a restricted key to an exact list, and `--permission-mode` picks between `all`, `readonly` and `restricted`. The first two repeat. On `update`, the bindings you pass replace the key's existing ones rather than adding to them. ```bash theme={null} langwatch api-keys create \ --name "Nightly evaluations" \ --permission-mode restricted \ --binding CUSTOM:PROJECT:proj_01HZX... \ --permission project:view \ --permission evaluations:create ``` ### Organization Read and update the organization your key belongs to. There is no id to pass: the credential decides which organization you are talking to. ```bash theme={null} langwatch organization get langwatch organization update --name "Acme" --support-contact support@acme.example # Presence and trace sharing are on/off pairs langwatch organization update --no-trace-sharing ``` Turning trace sharing off revokes the share links that were already handed out, not only future ones. ### Members Manage the people already in the organization. Someone joins through an invite, so `invites` is where a new person starts. ```bash theme={null} langwatch members list langwatch members list --include-disabled langwatch members get user_01HZX... langwatch members update user_01HZX... --role ADMIN # Offboarding: disable keeps the membership, remove does not langwatch members disable user_01HZX... langwatch members enable user_01HZX... langwatch members remove user_01HZX... ``` `langwatch members access user_01HZX...` answers the auditor's question: everything the person can reach, and whether it comes from their organization role, a group, or a binding of their own. ### Invites Invite people in, up to 50 at a time, landing on the teams you name. Accepting stays a browser step; everything before it is here. ```bash theme={null} # One person, one team langwatch invites create \ --email engineer@acme.example \ --role MEMBER \ --team team_01HZ...:MEMBER # A batch, from a file langwatch invites create --file ./new-hires.json langwatch invites list langwatch invites revoke ``` `--email` and `--team` repeat, and `--role` takes either one value for the whole batch or one per email. When people need different teams or a custom role each, `--json`, `--file` and `--stdin` take a JSON array of invites instead, the same entries the API takes. Every invite comes back with its acceptance link, so a deployment with no email provider still has something to send. ### Teams Teams group projects and the people who work on them. ```bash theme={null} langwatch teams list langwatch teams get team_01HZ... langwatch teams create --name "Engineering" langwatch teams update team_01HZ... --name "Platform Engineering" langwatch teams archive team_01HZ... langwatch teams members list team_01HZ... langwatch teams members add team_01HZ... user_01HZX... --role MEMBER langwatch teams members remove team_01HZ... user_01HZX... ``` ### Groups A group is a named set of people that carries role bindings, so everyone in it inherits the same access. Groups synced from your identity provider land here too. ```bash theme={null} langwatch groups list langwatch groups get group_01HZ... # Create with members and access in one call langwatch groups create \ --name "Backend Engineers" \ --member-id user_01HZX... \ --binding MEMBER:TEAM:team_01HZ... langwatch groups rename group_01HZ... --name "Platform Engineers" langwatch groups delete group_01HZ... langwatch groups members list group_01HZ... langwatch groups members add group_01HZ... user_01HZX... langwatch groups members remove group_01HZ... user_01HZX... langwatch groups bindings list group_01HZ... langwatch groups bindings add group_01HZ... \ --role MEMBER --scope-type TEAM --scope-id team_01HZ... langwatch groups bindings remove group_01HZ... ``` A group your identity provider owns cannot be renamed or have its membership edited here, because the provider is the source of truth for both. Its bindings are still yours to manage. ### Roles Custom roles are named permission sets, for when `ADMIN`, `MEMBER` and `VIEWER` are not the shape you need. ```bash theme={null} # The catalog the permission keys come from langwatch roles permissions langwatch roles create \ --name "Release reviewer" \ --description "Reads traces and runs evaluations, changes nothing" \ --permission project:view \ --permission traces:view \ --permission evaluations:create langwatch roles list langwatch roles get langwatch roles delete ``` `langwatch roles update` replaces the permission set with the `--permission` flags you pass, so send the full list you want the role to end up with. A role something still holds cannot be deleted: the error says how many bindings and assignments are in the way. ### Role bindings A binding is one sentence: this principal has this role, here. Users, groups and API keys are all principals. ```bash theme={null} langwatch role-bindings list langwatch role-bindings list --principal-type user --principal-id user_01HZX... langwatch role-bindings list --scope-type PROJECT --scope-id proj_01HZX... langwatch role-bindings create \ --principal-type group --principal-id group_01HZ... \ --role MEMBER \ --scope-type TEAM --scope-id team_01HZ... # A service credential, scoped to one project through a custom role langwatch role-bindings create \ --principal-type api-key --principal-id key_01HZX... \ --role CUSTOM --custom-role-id \ --scope-type PROJECT --scope-id proj_01HZX... langwatch role-bindings update --role ADMIN langwatch role-bindings delete ``` Creating a binding that already exists reports `role_binding_already_exists` rather than making a second one, so a provisioning script can run twice. Updating changes only the role a binding grants; the principal and the scope are the binding's identity, so moving a grant means deleting one binding and creating another. ### SCIM tokens The bearer tokens your identity provider presents to the SCIM endpoints. ```bash theme={null} langwatch scim-tokens create --description "Okta production" langwatch scim-tokens list langwatch scim-tokens revoke ``` The token value is printed once, when you create it. To rotate: create a second token, store it in the provider, watch the new token's last-used time move, then revoke the old one. ### Organizations (self-hosted) The one family that authenticates against the instance rather than an organization, because it runs before any organization exists. It is available on self-hosted deployments that have `LANGWATCH_INSTANCE_ADMIN_API_KEY` configured. ```bash theme={null} export LANGWATCH_INSTANCE_ADMIN_API_KEY=... langwatch organizations create --name "Acme" --slug acme langwatch organizations list langwatch organizations get organization_01HZ... ``` Creating an organization returns an organization admin API key alongside it, printed once. That key is what every command above takes, so an infrastructure-as-code run can go from an empty deployment to a configured organization without opening a browser. Pass the instance credential with `--instance-key` if you would rather not export it. ## Trigger experiments Experiments batch-run an agent or prompt against a dataset and produce an evaluation report: ```bash theme={null} langwatch evaluation --help ``` Typically you script this in CI: check out the branch, run the experiment, fail the build if the pass rate drops below your threshold. The CLI emits machine-readable results so this plumbing is straightforward. ## Progressive disclosure The CLI leans heavily on `--help`. Every subcommand has its own, and the top-level `langwatch --help` is the best way to discover what's available: ```bash theme={null} langwatch --help langwatch prompt --help langwatch prompt tag --help langwatch prompt tag create --help ``` New capabilities show up in `--help` the moment they ship, so you never have to wonder whether a flag exists. ## AI Gateway commands The CLI provisions AI Gateway resources without touching the UI. Behaviour matches the dashboard exactly; both share a server-side service layer. Every command takes `--format json` for scripting. ### Virtual keys ```bash theme={null} langwatch virtual-keys list langwatch virtual-keys get vk_01HZX9K3MN... # Create: the secret is printed once, store it immediately langwatch virtual-keys create \ --name prod-key \ --scope team:team_01HZ... \ --providers-allowed mp_01HZX...,mp_01HZY... \ --budget-limit 500 --budget-window month langwatch virtual-keys update vk_01HZX9K3MN... --budget-limit 750 langwatch virtual-keys spend vk_01HZX9K3MN... --from 2026-07-01T00:00:00Z # Rotate: the new secret is printed once, the old one keeps working for 24 hours langwatch virtual-keys rotate vk_01HZX9K3MN... # Disable is reversible, revoke is not langwatch virtual-keys disable vk_01HZX9K3MN... --reason "payment overdue" langwatch virtual-keys enable vk_01HZX9K3MN... langwatch virtual-keys revoke vk_01HZX9K3MN... ``` `--scope` takes `type:id` pairs (`org`, `team`, or `project`) and repeats for several; it defaults to the calling project. `--providers-allowed` is a comma-separated list of ModelProvider ids. The atomic `--budget-*` flags cap the key itself and accept `day`, `week`, or `month`. ### Gateway budgets ```bash theme={null} langwatch gateway-budgets list --scope-type team,project langwatch gateway-budgets create \ --name eng-team-monthly \ --scope team --team team_01HZ... \ --window month --limit 5000 --on-breach warn # Anchor the cycle to your own billing date instead of the calendar langwatch gateway-budgets create \ --name customer-acme \ --scope virtual-key --virtual-key vk_01HZX... \ --window month --limit 250 --cycle-anchor-at 2026-01-17T09:00:00Z langwatch gateway-budgets update bgt_01HZ... --limit 6000 langwatch gateway-budgets reset bgt_01HZ... --reason "contract renewed" langwatch gateway-budgets archive bgt_01HZ... ``` `--scope` is `organization|team|project|virtual-key|principal|group`, each paired with its own id flag. `--window` accepts `minute|hour|day|week|month|total|manual`, `--on-breach` is `block` (default) or `warn`, and `--limit` is USD. `--cycle-anchor-at` takes an RFC 3339 instant and cannot be used with `total` or `manual`; it is fixed at creation. List output colourises spent-vs-limit, red at 100%, yellow from 80%. ### Webhooks Endpoint management needs an **organization** API key (see [Webhooks](/docs/features/webhooks)). ```bash theme={null} langwatch webhooks list langwatch webhooks get whe_01HZ... langwatch webhooks event-types # the subscribable catalog, by family # Create: the signing secret is printed once langwatch webhooks create \ --url https://billing.example.com/webhooks/langwatch \ --events gateway.request.completed,gateway.request.settled langwatch webhooks update whe_01HZ... --max-batch-size 50 --max-in-flight 4 langwatch webhooks test whe_01HZ... # one signed test.ping through the real path langwatch webhooks health whe_01HZ... # how stale your feed could be langwatch webhooks deliveries whe_01HZ... --limit 100 langwatch webhooks roll-secret whe_01HZ... langwatch webhooks disable whe_01HZ... langwatch webhooks enable whe_01HZ... langwatch webhooks archive whe_01HZ... # The org's emitted-events log, independent of delivery langwatch webhooks events --type gateway.request.completed --from 2026-07-01 ``` `--events` replaces the whole subscription set. `deliveries` and `events` page with `--cursor` and `--limit`. ### Spend events The billing pull surface, also an **organization** API key (see [Billing and spend events](/docs/ai-gateway/billing-events)). ```bash theme={null} # Reconciliation checksums first langwatch spend-events summary --group-by virtual_key --from 2026-07-01 --to 2026-08-01 # Item diff only where a checksum diverged langwatch spend-events list --from 2026-07-01 --to 2026-08-01 --virtual-key vk_01HZX... # One external end user's rollup langwatch spend-events by-user customer-acme-42 --window month # Re-deliver a gap to one endpoint (7-day window cap) langwatch spend-events replay --from 2026-07-14 --to 2026-07-15 --endpoint whe_01HZ... ``` Date flags take ISO-8601 or epoch milliseconds. Provider credentials and cache rules have no CLI group. Manage providers with `langwatch model-provider`, and cache rules over [REST](/docs/ai-gateway/api/management#cache-rules) or the dashboard. Required token permissions map onto the [RBAC](/docs/ai-gateway/rbac) grants: | Command group | Permission | | | | | | | | --------------------- | -------------------- | --------------------- | -------------------- | ------ | ------------- | ----------------------- | ------------------------- | | \`projects list | get\` | `project:view` | | | | | | | `projects create` | `project:create` | | | | | | | | `projects update` | `project:update` | | | | | | | | `projects delete` | `project:delete` | | | | | | | | `api-keys list` | `organization:view` | | | | | | | | \`api-keys create | revoke\` | `organization:manage` | | | | | | | \`virtual-keys list | get\` | `virtualKeys:view` | | | | | | | `virtual-keys create` | `virtualKeys:create` | | | | | | | | \`virtual-keys update | disable | enable\` | `virtualKeys:update` | | | | | | `virtual-keys rotate` | `virtualKeys:rotate` | | | | | | | | `virtual-keys revoke` | `virtualKeys:delete` | | | | | | | | `virtual-keys spend` | `gatewayUsage:view` | | | | | | | | `gateway-budgets *` | `gatewayBudgets:*` | | | | | | | | \`webhooks list | get | deliveries | health | events | event-types\` | `webhookEndpoints:view` | | | \`webhooks create | update | archive | roll-secret | test | enable | disable\` | `webhookEndpoints:manage` | | `spend-events *` | `gatewaySpend:view` | | | | | | | See the [public REST API reference](/docs/ai-gateway/api/chat-completions) for direct HTTP calls that don't require Node. ## Agent usage In a terminal the CLI prints tables, colour, and spinners. Driven by an AI coding assistant it switches to machine output automatically. Everything an agent needs to know is also built into the CLI itself: `langwatch help agent-mode` prints the condensed version of this section. ### Agent mode `--agent` on any command, or auto-detection from the environment (`CLAUDECODE`, `CLAUDE_CODE`, `CURSOR_AGENT`, `GITHUB_COPILOT`, `AMAZON_Q`, `LW_AGENT_MODE`, `LANGWATCH_AGENT_MODE`), switches output to compact single-line JSON and turns colour and spinners off: ```bash theme={null} langwatch trace search --agent LW_AGENT_MODE=1 langwatch monitor list ``` ### Output contract Every command accepts the same output flags (with a few documented exceptions: `trace export -o` is an output *file*, the coding-assistant wrappers pass flags through to the wrapped tool, and `dataset records add/update --json` takes a record *payload*): | Flag | Meaning | | ----------------------- | ---------------------------------------------------------------------------------------- | | `-o, --output ` | `table` (human default), `json`, `agents` (compact single-line JSON), `yaml` | | `--json ` | JSON with only the given comma-separated fields | | `--jq ` | Project the result before printing: dot paths, `.items[]`, `.items[].field`, `\| length` | ```bash theme={null} langwatch trace search --jq '.traces[].traceId' langwatch evaluator list --json id,name,slug langwatch monitor list -o yaml ``` The legacy `-f/--format json` spelling keeps working and maps onto the same contract. The contract is fully wired for traces, evaluators, monitors, status, skills, `commands`, and `help-tree`; on the remaining resource commands the flags parse but machine output is still rolling out, so those commands keep their legacy `-f json` behavior until migrated. ### Structured errors A failed command prints one JSON document on **stdout** (`{ "ok": false, "error": { "code", "message", "httpStatus", "suggestions", "docUrl", "traceId", ... } }`), keeps the human-readable block on **stderr**, and exits non-zero. Never merge the streams (`2>&1`) when parsing output; hints and error prose live on stderr by design so stdout stays parseable. ### Discovery An agent can learn the whole CLI without reading these docs: ```bash theme={null} langwatch commands --flat --jq '.commands[].path' # machine catalog: args, flags, hints, token costs langwatch help-tree # compact annotated tree for context injection langwatch help agent-mode # the agent-mode guide, from the CLI itself langwatch status # project overview + what needs attention ``` ### Skills The CLI carries LangWatch's agent skills and installs them into `~/.agents/skills` (or the project-level `.agents/skills` with `--dir .`): ```bash theme={null} langwatch skills list # bundled skills + installed state langwatch skills install --all # install everything (use --dry-run to preview) langwatch skills get tracing # raw SKILL.md on stdout, for piping into context ``` See the [Skills Directory](/docs/skills/directory) for what each skill covers. ### Daemon note Non-TTY invocations (agents, pipes, CI) are served by the background daemon described below. The output is identical, and the command returns faster. `LANGWATCH_NO_DAEMON=1` opts out per invocation. ## Report issues to LangWatch If anything did not work (broken commands, confusing docs, unexpected errors, things that took trial and error), send it straight to the LangWatch team. No login or API key needed: ```bash theme={null} # Summary report: what you tried, verbatim errors, what you had to figure out npx langwatch report --user-approved \ --title "scenario create 500" \ --summary "langwatch scenario create returned HTTP 500 with ..." # Full session report (best): attach the transcript, redacted locally npx langwatch report --user-approved \ --title "agent stuck instrumenting" \ --session ~/.claude/projects//.jsonl ``` Agents must ask the user for permission first, then pass `--user-approved`. Secrets, API keys, emails, and phone numbers are redacted locally before anything is sent; the [redaction rules are auditable](https://github.com/langwatch/langwatch/blob/main/packages/redaction/src/sessionReport.ts) and `--dry-run` previews the exact payload. See the [reporting guide](/docs/support#reporting-issues-from-coding-agents) for transcript locations and details. ## The background daemon Non-interactive invocations (an agent piping output, CI, any call whose stdin/stdout/stderr is not a TTY) are served by a warm background daemon instead of paying node's cold start on every call. Interactive (terminal) invocations always run in-process and are unaffected, as are commands that mutate auth or take over stdio (`login`, `logout`, `config`, `open`, `request-increase`, `init-shell`, `report`, the `claude`/`codex`/`cursor`/`gemini`/`opencode` wrappers, `daemon` itself) and long-running flags (`--follow`, `--watch`). Windows is excluded too: the daemon is not supported there, so on Windows every invocation runs in-process. What to know: * **One daemon per identity.** The socket name is a hash of endpoint + API key + uid + config path, so two projects with different keys never share a daemon. The socket lives in `$XDG_RUNTIME_DIR` (or the temp dir) with `0600` permissions inside a `0700` directory, and the daemon self-exits after 10 idle minutes. * **Always optional.** If no daemon is reachable, or it is stale, or from an older CLI build, the command runs in-process, and a daemon is auto-spawned in the background for next time. Set `LANGWATCH_DAEMON_NO_SPAWN=1` to disable the auto-spawn. * **Buffered output.** A daemon-served command's output is buffered and flushed when it exits, so a mid-command daemon failure can retry in-process without duplicating output. Piped callers therefore see output at exit, not progressively (beyond an 8MB buffer it starts streaming early). * **Timeouts and cancellation.** A command that hangs is abandoned after a per-request timeout (default 10 minutes, override with `LANGWATCH_DAEMON_REQUEST_TIMEOUT_MS`) with exit code 124; the clock starts when the daemon accepts the request, so time spent queued behind another caller's working directory counts against it. Ctrl-C on the client cancels the server-side command with exit code 130. * **Environment hygiene.** The daemon does not inherit the spawning shell's environment. It receives only process essentials (PATH, HOME, locale, …), the pinned identity variables, and the spawner's allowlisted output/proxy knobs (`FORCE_COLOR`, `NO_COLOR`, `HTTPS_PROXY`, …). Those allowlisted values from the *spawning* caller persist in the daemon's baseline and are visible to later callers that don't set them; secrets are never in the allowlist. Per request, the caller's own `.env` is applied scoped to `LANGWATCH_*` keys only. Manage it explicitly: ```bash theme={null} langwatch daemon status # pid, uptime, served/in-flight counts (--json supported) langwatch daemon start # start in the background (normally automatic) langwatch daemon stop # stop and remove the socket ``` Opt out: ```bash theme={null} LANGWATCH_NO_DAEMON=1 langwatch trace list # per-invocation langwatch config set daemon off # persistent, user-global (~/.langwatch/config.json) ``` `config set daemon off` doesn't stop an already-running daemon. Clients stop dialling it, and it idles out on its own within 10 minutes (`langwatch daemon stop` kills it immediately). ## Use the CLI as your agent's control plane An AI coding assistant operates LangWatch end-to-end through the CLI. Skills like [Tracing, Evaluations, Scenarios, and Prompt Versioning](/docs/skills/directory) are built on top of it, the assistant reads docs via `langwatch docs`, runs platform operations via the subcommands, and verifies its own work by searching traces and inspecting simulation runs. If you're building your own agent workflows, the pattern works the same way: give the assistant the CLI and ask it what you want. Everything it does is auditable in the LangWatch app afterwards. **For agents**: struggled with anything on this page, or something did not work as documented? Ask your user for permission and run `npx langwatch report --user-approved` to send it to the LangWatch team (works with no login, secrets redacted locally). It is how rough edges get fixed for every agent. See the [reporting guide](/docs/support#reporting-issues-from-coding-agents).