Introducing Langy: Your Automated AI Engineer

Today we are launching Langy, an AI engineer that works inside LangWatch. It is a full coding agent: it reads your production traces, writes Scenario tests and evaluations against what it finds, connects to your GitHub, opens a pull request on your actual codebase, and proves the change by running the simulations it wrote in your CI. You talk to it where your evaluation results already live, and what comes back is a diff with green checks attached.

The first thing we did with it: we plugged Langy into a robot over MCP and asked it to test our own customer-support voice agent. It kicked off the Scenario simulations itself.

AI has pushed software into work it never used to touch. Agents are doing legal work, health work, financial work, and the engineers building them are not lawyers, doctors, or bankers. The domain expert matters more to a software team than ever, and yet on most teams they still cannot contribute a change: any agent valuable enough to do that work is complex, with an evaluation pipeline and a scenario suite behind it, and improving it means touching code and having the AI engineering expertise to do it well.

Every platform in this market, ours included, promised that collaboration. In practice the domain experts still collaborate through spreadsheets and Slack messages, and the engineers became the bottleneck for every hypothesis anyone else wants to try. Engineers do not want that job either; nobody becomes an AI engineer to translate other people's prompt tweaks.

Langy is our answer. What changed to make it possible is that coding agents got genuinely good, while LangWatch grew into the platform where agent testing, evaluation, and improvement already happen. When your agents get complex, the work lives in two places at once, the platform and the codebase, and Langy is at home in both. A PM or domain expert describes what the agent should do differently, in plain language. Langy turns that into scenario tests and evaluations, runs them, drafts the fix, and opens the pull request. An engineer approves a diff instead of translating a wish, and the CI run on that PR shows the simulations Langy wrote, passing or failing, so the review is about the change rather than about trust.

Concretely, Langy can:

• Read through thousands of production conversations to find the patterns in how your agent actually behaves, and cluster them into the problems worth fixing.
• Write the evaluations and Scenario simulations that pin that behavior down, then run them to see where the agent breaks.
• Reproduce a single bad trace as a failing test you can keep.
• Connect to your GitHub and open a pull request that fixes the agent, with the change explained.
• Prove the fix by running its own simulations in your CI before anyone merges.
• Handle the rest of the loop too: compare two model or prompt versions, translate prompts across languages, and set up the signals that catch a problem coming back.

Everyone on the team gets something out of that. PMs and domain experts get agency, pun intended: they find the problems, improve the prompts, add the scenarios their domain actually needs. Engineers get domain expertise landing as reviewable pull requests instead of Slack messages, and keep their focus for the genuinely hard problems. And Langy speeds engineers up on the platform too: finding that one trace, spotting the pattern behind it, pulling the metric or the visualization that explains what the agent has been doing in production or in last week's experiment, at a glance instead of an afternoon.

That last part scales further than a person can. Point Langy at a few thousand production traces and ask what is going on: it clusters the topics, sets up signals for the patterns worth watching, and comes back with findings and a proposal for what to do about each. The alternative was an engineer sampling a handful by hand and extrapolating.

"AI engineer" is a crowded claim this year, and skepticism is the right reaction to one more of them. Most of what ships under the name is a chat box on top of dashboards. Langy has hands, and it shows its work: a pull request on a real repo and a green CI run on tests it wrote itself.

This week we will be launching videos, tutorials, and more on how to use Langy. Stay tuned.

One detail we like: every Langy run is itself an agent run, traced, costed, and evaluated inside LangWatch like any other, so we watch Langy work with the same tools you will use to watch yours.

Langy is rolling out to LangWatch teams now. If you want it on your agent, book a demo and we will set it up with you.