@langwatch/scenario
    Preparing search index...

    Function evaluator

    • Attaches a LangWatch evaluator to a scenario run.

      Parameters

      • evaluatorRef: string

        A built-in type such as ragas/sql_query_equivalence, or a saved evaluator as evaluators/<slug>.

      • options: Omit<ScenarioEvaluator, "evaluator"> = {}

        Whether the evaluator gates the run, its input mappings and its settings overrides. A mapping is a function of the scenario state or a literal. Unmapped inputs are inferred from their names.

      Returns ScenarioEvaluator

      scenario.evaluator("ragas/sql_query_equivalence", {
      required: true,
      mappings: {
      output: (state) => state.toolCalls("run_sql").last?.input,
      expected_output: scenario.field("golden_sql"),
      },
      });