ClaudeCodeAgentAdapter

Drives the Claude Code CLI (claude -p --output-format stream-json) as an agent-under-test inside a Scenario run. Because it spawns the real binary, skill loading, CLAUDE.md discovery, and permission prompts behave exactly as they do in production — making it the right tool for testing Claude Code skills and subagents end-to-end.

Placement note: This page lives under agent-integration/ rather than voice/adapters/. The decision to back this adapter with the CLI (not the Anthropic SDK) is intentional — see PR / issue #686 for rationale. Maintainers may relocate this page once a canonical non-voice adapter section is established.

:::info TypeScript only

The Claude Code adapter is TypeScript-only today. Python parity is tracked as a follow-up.

:::

Constructor

claudeCodeAgent(config) is the recommended factory. It wraps new ClaudeCodeAgentAdapter(config) and additionally injects the skill (if skillPath is set) before the adapter is returned.

Configuration

Skill testing

The adapter's distinguishing feature is first-class skill testing:

Injection via skillPath — pass an absolute path to a SKILL.md and the factory will copy it into <workingDirectory>/.skills/<skill-name>/SKILL.md and point the working directory's CLAUDE.md at it, so Claude reads the skill before doing anything else. A CLAUDE.md the fixture already ships is kept: only the skills it does not mention yet are appended, and the instruction is written once. The injected layout matches Claude Code's native discovery conventions.

Skills installed some other way: when your harness copies skills into .skills/ itself (several skills, a rendered SKILL.md, a fixture project), call pointClaudeMdAtSkills(workingDirectory) after the copy to get the same CLAUDE.md instruction for every skill it finds there.

import { pointClaudeMdAtSkills } from "@langwatch/scenario";
 
installMySkills(workingDirectory); // writes .skills/<name>/SKILL.md
pointClaudeMdAtSkills(workingDirectory);

Asserting the skill was read — after the scenario run, call assertSkillWasRead(state, skillName) to verify that the skill file actually appeared in the conversation. It throws (naming the skill) if no read evidence is found, catching cases where the agent hallucinated instructions rather than reading the skill.

import { assertSkillWasRead } from "@langwatch/scenario";
 
// inside your test, after scenario.run():
assertSkillWasRead(result, "my-skill");

skillName is the parent directory name of the SKILL.md — i.e. the segment in .skills/<skillName>/SKILL.md.

Reading what the agent ran: with output: "messages" (below), the shell commands the agent ran are tool-call parts naming the Bash tool. bashCommands(state) lists them, and only them: a command quoted in the skill text or in the agent's own explanation is not a command that ran.

import { bashCommands } from "@langwatch/scenario";
 
const commands = bashCommands(result);
expect(commands.some((c) => c.startsWith("langwatch prompts sync"))).toBe(true);

Structured output

By default a turn comes back as text, with each tool call and result rendered as a readable line (Tool Called: Bash({"command":"ls"}), Tool Result: ...). With output: "messages" a turn comes back as AI SDK messages instead: an assistant message with a text part and one tool-call part per tool the agent called, and a tool message with a tool-result part per result. The judge and the user simulator then read the calls structurally, and the LangWatch run view renders them as tool calls rather than as prose.

claudeCodeAgent({
  workingDirectory,
  output: "messages",
  maxToolResultChars: 4000,
});

Both renderings cap what a tool result and a tool call input contribute to the conversation (maxToolResultChars, maxToolInputChars). The judge reads the whole conversation on every step, and one exported trace or log file in a tool result would push a long run past its context window for a reason that has nothing to do with the agent under test. A result keeps its first characters, where command names, ids and URLs stand; a call input keeps far more, since it is what the agent wrote and is the work under judgement. The dropped count is appended so the judge knows the text is cut. Thinking blocks are never part of either rendering, and a block the conversation has no part for (an image, a document) leaves a line naming it instead of disappearing.

toModelMessages(rawMessages, limits?) and parseStreamJson(stdout) are exported for a harness that reads a stream-json transcript on its own.

Environment

The CLI inherits this process's environment. env applies on top of it, and a key set to undefined is removed, which is how a test keeps a provider key for its judge while hiding it from the agent, or keeps the batch id of the harness out of the scenario runs the agent writes itself:

claudeCodeAgent({
  workingDirectory,
  env: {
    PATH: `${localBin}:${process.env.PATH}`,
    OPENAI_API_KEY: undefined,
    SCENARIO_BATCH_RUN_ID: undefined,
  },
});

FORCE_COLOR=0 is always set last, so the transcript carries no escape codes.

Process lifecycle

A claude -p run spawns its own children: the shell commands of its Bash tool, a test runner, a dev server. When the process that spawned Claude dies with a turn still running (a test runner killed under memory pressure, a Ctrl-C, a session restart), nothing would tell those processes to stop. They would reparent to pid 1 and keep running, burning tokens and memory into a pipe nobody reads. The adapter takes three measures so that does not happen:

• The CLI is spawned as the leader of its own process group, so a kill aimed at the group reaches every descendant, not only the CLI. The timeout kill is such a group kill: SIGTERM to the group, then SIGKILL two seconds later.
• On the harness's own exit every group still running gets SIGKILL. That covers a normal exit, process.exit() and an uncaught exception.
• A small shell watchdog (claude-code-watchdog in ps), detached from both, polls the harness pid and the CLI pid once a second and terminates the CLI's group when the harness is gone. That covers SIGKILL and any other death that runs no JavaScript. The watchdog exits by itself the moment the CLI exits, so a healthy turn leaves nothing behind.

Process groups and the watchdog are POSIX. On Windows the CLI is spawned as before and only the exit hook applies.

Error handling

call() rejects with ClaudeCodeCliError when the CLI exits non-zero or is killed by a signal — auth failures, rate limits, an unknown model, and similar unrecoverable errors all surface this way. That's what you get calling call() directly; through scenario.run(...) — the common path — the executor wraps every adapter error in a plain Error, so catch that and read the original off .cause:

import { ClaudeCodeCliError, LostSessionError } from "@langwatch/scenario";
 
try {
  await scenario.run({ /* ... */ });
} catch (err) {
  // scenario.run wraps adapter errors — the original is on `.cause`
  const cause = err instanceof Error ? err.cause : undefined;
  if (cause instanceof LostSessionError) {
    // the --resume session vanished; sessionId / threadId name the dead session
    console.error("lost session", cause.sessionId, cause.threadId);
  } else if (cause instanceof ClaudeCodeCliError) {
    console.error(cause.exitCode, cause.signal, cause.stderr);
  }
}

exitCode (number | null) and signal (NodeJS.Signals | null) mirror Node's child_process exit info; stderr (string) is the CLI's captured output. subtype (string | undefined) is the CLI's machine-readable failure class off its terminal result envelope (e.g. "error_during_execution"), and errors (string[] | undefined) is the structured errors[] array the CLI fielded on that envelope.

A vanished --resume session rejects with LostSessionError, a ClaudeCodeCliError subclass that additionally exposes the dead sessionId and its threadId and sets the originating ClaudeCodeCliError as .cause. Because it is a subclass, instanceof ClaudeCodeCliError still matches it while instanceof LostSessionError singles it out — see Lost session. (It is thrown only under the default replayOnLostSession: false; with replay on, the turn rebuilds instead of throwing.)

:::warning stderr and errors are unredacted CLI output

stderr and every entry of errors[] carry whatever the CLI wrote verbatim — they may include sensitive detail (environment values, a rejected API key echoed in an error). Both are enumerable (as is subtype), so they appear in JSON.stringify(error) too, and errors[] is additionally embedded into the error's message (falling back to stderr when the envelope carried no errors[]). Do not log them verbatim to shared or public sinks; redact or drop them before they leave a trusted boundary.

:::

:::warning Error messages are exported to tracing by default

When an agent.call() fails inside scenario.run(...), the error message — which embeds the CLI's errors[] (or stderr) verbatim — is exported to whatever tracing backend LangWatch is configured with: the span handling calls recordException(err) and setStatus({ message: err.message }) with no redaction and no opt-out. Any sensitive detail in the CLI's error output therefore reaches your tracing sink on every failed turn. A scrubbing hook is tracked in #754.

:::

Caveats

• claude must be on PATH (or pointed to via claudeBin / CLAUDE_BIN). The adapter does NOT install the CLI automatically — it is not an npm dependency. Install it separately: claude.ai/code.
• stream-json format stability — the adapter parses the CLI's --output-format stream-json output. Output shape can change across CLI versions; pin the CLI version in CI if you need reproducibility.
• --dangerously-skip-permissions — skipPermissions: true runs the agent without any permission prompts. Only enable this in sandboxed or CI environments where you control the working directory.
• Session continuation — per threadId, the first turn sends the full history with no --resume; later turns pass --resume <session_id> and send only the new messages.
• Lost session (default: fail loudly) — if the CLI reports that a resumed session no longer exists, the adapter evicts the dead id and, by default, rejects the turn with an actionable error naming replayOnLostSession. Rejecting is the default on purpose: the only alternative is to rebuild the turn by re-flattening the whole transcript into a fresh session, which loses server-side session state (context cache, tool state) — a real modality change that would silently alter what a run measures. Set replayOnLostSession: true to opt into that rebuild; then a single call() can spawn the CLI twice (the doomed resume, then the replay) and warns via the injected logger. Any other CLI failure — auth, rate limit, unknown model, a signal, a timeout — rejects with ClaudeCodeCliError (see Error handling), never evicts the session, and is not retried.