Skip to main content
POST
Run an evaluator

Authorizations

X-Auth-Token
string
header
required

Project API key for sending traces and accessing project-scoped resources. Format: sk-lw-... (no underscore). Obtain one by creating a project via the Admin API or the LangWatch UI.

Path Parameters

evaluator
string
required

Which evaluator to run. Either a built-in id (ragas/faithfulness), the slug of a monitor configured in this project, or evaluators/{slug|id} for a saved evaluator. GET /api/evaluations/list returns the built-in ids.

Body

application/json
data
object
required

What the evaluator scores. Which fields are required depends on the evaluator; its own entry under Built-in Evaluators lists them.

settings
object

Per-call overrides of the evaluator's settings. Anything omitted falls back to the saved evaluator or monitor, then to the evaluator's own defaults.

trace_id
string | null

Attaches the result to a trace you already sent

evaluation_id
string | null

Supply your own id to make the call idempotent

evaluator_id
string | null
name
string | null

Overrides the name the result is recorded under

as_guardrail
boolean | null

Evaluate as a guardrail: a skipped or failed evaluation answers passed rather than an error, so a caller can gate on one field. The /api/guardrails path sets this for you.

Response

The evaluator ran, declined, or failed. Branch on status; in guardrail mode passed is set on all three.

status
string
required
Allowed value: "processed"
score
number
passed
boolean
label
string
details
string
cost
object

What running the evaluator cost

raw_response
any

The evaluator's own output, unprocessed