Automatically build datasets from real-time traces

You can keep continously populating the dataset with new data arriving from production by using Automations, mapping trace fields to any dataset columns you prefer. Simply go to the Messages page and select a filter (for example, by model), the Add Automation button will be enabled:

For Action, select Add To Dataset, and chose the right fields to map from the trace to the dataset:

Hit save, and that’s it! Now every time a new message matches the filter, the automation will be fired and the dataset will be populated with the new row.

Generating a dataset with AI Add trace threads to datasets

Get Started

Agent Simulations

Observability

Evaluations

Prompt Management

Platform

Examples & Cookbooks

Help