Launch Week - Day 3: Testing Voice Agents with LangWatch Scenario in Real Time

LangWatch Scenario now supports testing voice-powered AI agents using OpenAI's Realtime API through a new RealtimeAgentAdapter. This enables comprehensive end-to-end testing of voice conversations, including voice-to-voice interactions, audio processing, and real-time agent behavior validation—all within the Scenario testing framework.

Developers can now ensure their voice agents perform reliably across different conversation scenarios, with the same session architecture used in both browser clients and automated tests.

Realtime Agent Testing

• Test real OpenAI Realtime API agents (not mocks) using the Scenario framework

• Validate voice-powered agent behavior through automated test scenarios

• Use the same session creation pattern in both browser and test environments

Voice-to-Voice Conversation Testing

• Run automated voice-to-voice scenario tests with audio input/output

• Test multi-turn conversations with realistic user simulation

• Capture and validate audio responses alongside transcripts

Audio Processing & Validation

• Process audio messages through the Realtime API in test scenarios

• Validate agent responses to both text and audio inputs

• Monitor audio events and response timing in automated tests

Framework Integration

• Seamlessly integrate realtime agents into existing Scenario test suites

• Combine with other agent types (text-based, tool-calling) in the same scenario

• Leverage Scenario's evaluation and judging capabilities for voice agents

Video Demo

Turn the volume up to listen in as the audio messages arrive!

The interactive demo showing the voice-powered Scenario expert interface with real-time conversation capabilities

🧑‍💻 Link to Code Example