# Introducing Scenario: Use an Agent to Test Your Agent

Scenario is an automated testing library for LLM agents that simulates real user interactions end-to-end.

*By Rogerio Chaves · April 8, 2025*

Canonical: https://langwatch.ai/blog/introducing-scenario-use-an-agent-to-test-your-agent

![Introducing Scenario: Use an Agent to Test Your Agent](https://framerusercontent.com/images/knXIpAu2qfHSHsySe2ZG8CGd0Y.webp)

> Scenario is now live on GitHub - [check it out](http://github.com/langwatch/scenario) and give it a ⭐️

Agent development today is stuck in a loop. You tweak a prompt, run a pretend conversation, hope it behaves better… and repeat. For every minor improvement, you risk breaking something else. It’s slow, manual, and fragile.

We’ve been there. So we built **Scenario**.

Scenario is an [open-source](http://github.com/langwatch/scenario) testing library that uses an AI agent to **test your AI agent**, eliminating the tedious back-and-forth of manual testing and making sure your agent works in real-world scenarios - not just in your head.

## The Problem: Agent Development is broken

If you've built an AI agent, you know the cycle:

-   Change a prompt ✍️
    
-   Talk to your agent pretending to be a user 💬
    
-   See if it works better 🤔
    
-   Repeat dozens of times 😫
    

This manual testing approach is time-consuming, non-repeatable, vibey-only and doesn't scale as your agent grows more complex. Without a way to automatically verify that each enhancement doesn't break existing functionality, agents become fragile and unpredictable. As a result, teams stop touching the code and the prompt, and stop moving forward.

## How Scenario works

Scenario transforms agent testing by simulating how real users interact with your agent. Instead of writing rigid input-output tests that break with every prompt change, you define high-level scenarios and success criteria. Our testing agent then dynamically converses with your agent until goals are met or failures are detected.

For example, here's how you would test a complex code assistant or website builder agent, like Cursor or Lovable:

```python
def test_generate_landing_page():
    scenario = Scenario(
        name="build landing page",
        user_goal="Generate a landing page with pricing table and hero section",
        success_criteria=[
            "Includes a pricing table",
            "Includes a hero section",
            "Outputs valid HTML"
        ]
    )
    run_scenario(scenario)
```

As you run it, you'll see the testing agents talking to your agent as if it was a user. It's mind-blowing.

## Why You'll Love It

-   **Natural Interactions**: Testing agent simulates real user behavior with natural variation
    
-   **End-to-End Testing**: Tests the entire agent experience, not just isolated components, allowing for refactoring with confidence
    
-   **Fast Feedback Loop**: Quickly identify when changes break existing functionality
    
-   **Debug Mode**: Watch conversations unfold in slow motion and intervene for debugging
    
-   **Feels Like Standard Testing**: Integrates seamlessly with pytest and your existing CI/CD pipeline
    
-   **Deterministic When Needed**: Caching options for reproducible tests during development
    

## Real-World Examples

Scenario excels at testing complex agent interactions:

-   **Support bots**: Ensure your agent handles common support requests correctly on long multi-turn conversations in various scenarios
    
-   **Travel bookings**: Test that your agent collects all required information and helps the user book hotel, flight and car through various scenarios
    
-   **Coaching agents**: Verify that the agent is able to coach the user through a large range of topics and conversations that may happen
    
-   **Swarm of agents**: Test that all the agents coordinate to complete multiple complex tasks
    
-   **Building assistants**: Validate your agent generates working code or rules and modify them as user requests mount
    

##### Install Scenario with pip:

```shell
pip install scenario-testing
```

Create your first scenario, and you're ready to test your agent with a single pytest command:

```shell
pytest test_my_agent.py -v
```

### More Than Just Testing

Scenario helps you:

-   **Document behavior**: Scenarios serve as living documentation of how your agent should behave
    
-   **Prevent regressions**: Catch unintended side effects of prompt or model changes
    
-   **Evolve with confidence**: Add new features knowing existing functionality remains intact
    

## Case Study: The Lovable Clone

For the launch of Scenario we wrote our own Lovable agent clone. If you are not familiar, Lovable is a website that allows you to create websites with a prompt. This is very powerful, but building a website is complex, for it to work well you need a very good prompt and then let the agent "run free" and deciding which tools to use.

Scenario then really helped us developing this Lovable clone, in this video, I show how it works and how can we extend it further while fully testing it with Scenario:

[Watch the video](https://www.youtube.com/embed/f8NLpkY0Av4?iv_load_policy=3&rel=0&modestbranding=1&playsinline=1&autoplay=0&mute=1)

You can check out the full example on the GitHub examples folder too.  
  

## Try It Today

Visit GitHub for full examples:

-   🔁 Multi-turn tool-using agents
    
-   🐞 Debug mode
    
-   ⚡ Parallel testing
    

> 👉 Star Scenario on GitHub 👉 Read the docs

**Scenario** is your agent's test suite. Use an agent to test your agent - and ship with confidence.