Skip to main content
A voice test is a scenario played out over audio. A simulated caller speaks to your agent, listens to each reply and answers, turn by turn. A judge then scores the call against your criteria. You write the scenario the same way as for a text agent. A voice scenario: a simulated caller, the agent under test and the judge This page gets one voice test running from the platform, against an ElevenLabs agent or a phone number. To test another transport, or to run voice tests in CI, see Test from code.

​
Before you start

Voice agents are in private beta. On cloud, write to support@langwatch.ai to enable them for your organization. Self-hosted, set FEATURE_FLAG_FORCE_ENABLE=release_voice_agents_enabled on the app.
Add the providers under Settings > Model Providers. See Model Providers.
  • OpenAI, for every voice test. The simulated caller speaks with OpenAI text-to-speech, and the judge reads an OpenAI transcript of the call.
  • ElevenLabs, to test an ElevenLabs agent. The key must be in the same ElevenLabs workspace as the agent, with convai_read, convai_write and conversation read.
  • Twilio, to test a phone number. The call is placed from your Twilio number.

​
Add your voice agent

  1. Open Agents in the sidebar and press New Agent.
  2. Pick Voice Agent and give it a Name.
  3. Under Reached via, pick ElevenLabs agent and paste the Agent id from ElevenLabs. For a phone number, pick Phone number and enter it in international format, for example +14155550123.
  4. Press Create Agent.
LangWatch tests the agent as ElevenLabs has it configured. It does not send prompt overrides. For an ElevenLabs agent, press Talk to it to hear the agent from your browser before you write a scenario. This call is not scored.

​
Run a scenario against it

Open Agent Testing, create a scenario the way you would for a text agent, and press Run. In the run dialog, pick your voice agent as the Agent to be tested. See Test from the platform for writing scenarios and the run dialog. A voice run takes one to two minutes per scenario, because every turn is spoken, then transcribed.

​
Read the result

The run shows the same verdict as a text run: each criterion passed or failed, with the judge’s reasoning. The conversation drawer adds the recording of the call, with the transcript per turn. See Results.

​
Test from code

The Scenario SDK runs voice tests from your own test runner, in Python or TypeScript. It has adapters for ElevenLabs, OpenAI Realtime, Gemini Live, Pipecat and Twilio, and can add background noise and interruptions. With LANGWATCH_API_KEY set, each run appears in Agent Testing with its recording and transcript.

Voice quick start

Install, set up keys and run your first voice scenario.

Choose an adapter

Pick the adapter for your transport.

Voice recipes

Audio effects, interruptions, multi-turn calls and observability.

Troubleshooting

The failures that come up most often in voice scenarios.

​
Reference

The caller voice is part of the scenario, so every run of it uses the same caller. In the scenario editor, open Customize scenario, then Caller voice.
Run a single scenario and, in the run dialog, pick Call it myself to talk to an ElevenLabs agent from your browser. The call is saved as a run of the scenario, and the judge scores it like a simulated call.The browser needs HTTPS or localhost and microphone permission.
A call is cut after 5 minutes. The run is marked “Cut at the call limit”, and the judge scores what was said. Each project runs 2 voice calls at a time per worker process, and more voice runs wait in the queue. Self-hosted, change these limits with the voice variables in Environment Variables.Every call uses ElevenLabs conversation minutes, or Twilio minutes for a phone number. A simulated call also uses OpenAI audio tokens.Projects that share one ElevenLabs key share that workspace’s agents and calls, because ElevenLabs checks the key, not the project. Give each project its own key to keep them apart.
Last modified on September 28, 2026