import requests
url = "https://app.langwatch.ai/api/evaluations/{evaluator}/evaluate"
payload = { "data": {} }
headers = {
"X-Auth-Token": "<api-key>",
"Content-Type": "application/json"
}
response = requests.post(url, json=payload, headers=headers)
print(response.text)const options = {
method: 'POST',
headers: {'X-Auth-Token': '<api-key>', 'Content-Type': 'application/json'},
body: JSON.stringify({data: {}})
};
fetch('https://app.langwatch.ai/api/evaluations/{evaluator}/evaluate', options)
.then(res => res.json())
.then(res => console.log(res))
.catch(err => console.error(err));curl --request POST \
--url https://app.langwatch.ai/api/evaluations/{evaluator}/evaluate \
--header 'Content-Type: application/json' \
--header 'X-Auth-Token: <api-key>' \
--data '{
"data": {}
}'{
"status": "<string>",
"score": 123,
"passed": true,
"label": "<string>",
"details": "<string>",
"cost": {
"currency": "<string>",
"amount": 123
},
"raw_response": "<unknown>"
}{
"error": "<string>",
"kind": "<string>",
"meta": {}
}{
"error": "<string>",
"kind": "<string>",
"meta": {}
}{
"error": "<string>",
"kind": "<string>",
"meta": {}
}{
"error": "<string>",
"kind": "<string>",
"meta": {}
}Run an evaluator
Run one evaluator over a single input and get its score back. Built-in evaluators whose id has two segments, such as ragas/faithfulness, are addressed with the two-segment form of this path. Bodies up to 30MB are accepted.
import requests
url = "https://app.langwatch.ai/api/evaluations/{evaluator}/evaluate"
payload = { "data": {} }
headers = {
"X-Auth-Token": "<api-key>",
"Content-Type": "application/json"
}
response = requests.post(url, json=payload, headers=headers)
print(response.text)const options = {
method: 'POST',
headers: {'X-Auth-Token': '<api-key>', 'Content-Type': 'application/json'},
body: JSON.stringify({data: {}})
};
fetch('https://app.langwatch.ai/api/evaluations/{evaluator}/evaluate', options)
.then(res => res.json())
.then(res => console.log(res))
.catch(err => console.error(err));curl --request POST \
--url https://app.langwatch.ai/api/evaluations/{evaluator}/evaluate \
--header 'Content-Type: application/json' \
--header 'X-Auth-Token: <api-key>' \
--data '{
"data": {}
}'{
"status": "<string>",
"score": 123,
"passed": true,
"label": "<string>",
"details": "<string>",
"cost": {
"currency": "<string>",
"amount": 123
},
"raw_response": "<unknown>"
}{
"error": "<string>",
"kind": "<string>",
"meta": {}
}{
"error": "<string>",
"kind": "<string>",
"meta": {}
}{
"error": "<string>",
"kind": "<string>",
"meta": {}
}{
"error": "<string>",
"kind": "<string>",
"meta": {}
}Authorizations
Project API key for sending traces and accessing project-scoped resources. Format: sk-lw-... (no underscore). Obtain one by creating a project via the Admin API or the LangWatch UI.
Path Parameters
Which evaluator to run. Either a built-in id (ragas/faithfulness), the slug of a monitor configured in this project, or evaluators/{slug|id} for a saved evaluator. GET /api/evaluations/list returns the built-in ids.
Body
What the evaluator scores. Which fields are required depends on the evaluator; its own entry under Built-in Evaluators lists them.
Show child attributes
Show child attributes
Per-call overrides of the evaluator's settings. Anything omitted falls back to the saved evaluator or monitor, then to the evaluator's own defaults.
Show child attributes
Show child attributes
Attaches the result to a trace you already sent
Supply your own id to make the call idempotent
Overrides the name the result is recorded under
Evaluate as a guardrail: a skipped or failed evaluation answers passed rather than an error, so a caller can gate on one field. The /api/guardrails path sets this for you.
Response
The evaluator ran, declined, or failed. Branch on status; in guardrail mode passed is set on all three.
Was this page helpful?