Get started with LangWatch Skills in seconds: Set up evals, scenario tests, and tracing just by asking your AI coding assistant.
Traces
Prompts
Annotations
Automations
Scenarios
Evaluators
- Overview
- POSTExact Match Evaluator
- POSTLLM Answer Match
- POSTBLEU Score
- POSTLLM Factual Match
- POSTROUGE Score
- POSTSQL Query Equivalence
- POSTLLM-as-a-Judge Boolean Evaluator
- POSTLLM-as-a-Judge Category Evaluator
- POSTLLM-as-a-Judge Score Evaluator
- POSTRubrics Based Scoring
- POSTRagas Answer Correctness
- POSTRagas Answer Relevancy
- POSTRagas Context Precision
- POSTRagas Context Recall
- POSTRagas Context Relevancy
- POSTRagas Context Utilization
- POSTRagas Faithfulness
- POSTRagas Faithfulness
- POSTRagas Response Context Precision
- POSTRagas Response Context Recall
- POSTRagas Response Relevancy
- POSTContext F1
- POSTContext Precision
- POSTContext Recall
- POSTAzure Content Safety
- POSTAzure Jailbreak Detection
- POSTAzure Prompt Shield
- POSTOpenAI Moderation
- POSTPresidio PII Detection
- POSTCustom Basic Evaluator
- POSTCompetitor Blocklist
- POSTCompetitor Allowlist Check
- POSTCompetitor LLM Check
- POSTOff Topic Evaluator
- POSTQuery Resolution
- POSTSemantic Similarity Evaluator
- POSTSummarization Score
- POSTValid Format Evaluator
- POSTLingua Language Detection
Saved Evaluators
import langwatch
df = langwatch.datasets.get_dataset("dataset-id").to_pandas()
experiment = langwatch.experiment.init("my-experiment")
for index, row in experiment.loop(df.iterrows()):
# your execution code here
experiment.evaluate(
"presidio/pii_detection",
index=index,
data={
"input": row["input"],
"output": output,
},
settings={}
)
[
{
"status": "processed",
"score": 123,
"passed": true,
"label": "<string>",
"details": "<string>",
"cost": {
"currency": "<string>",
"amount": 123
}
}
]Evaluators
Presidio PII Detection
Detects personally identifiable information in text, including phone numbers, email addresses, and social security numbers. It allows customization of the detection threshold and the specific types of PII to check.
POST
/
presidio
/
pii_detection
/
evaluate
import langwatch
df = langwatch.datasets.get_dataset("dataset-id").to_pandas()
experiment = langwatch.experiment.init("my-experiment")
for index, row in experiment.loop(df.iterrows()):
# your execution code here
experiment.evaluate(
"presidio/pii_detection",
index=index,
data={
"input": row["input"],
"output": output,
},
settings={}
)
[
{
"status": "processed",
"score": 123,
"passed": true,
"label": "<string>",
"details": "<string>",
"cost": {
"currency": "<string>",
"amount": 123
}
}
]Documentation Index
Fetch the complete documentation index at: https://langwatch.ai/docs/llms.txt
Use this file to discover all available pages before exploring further.
Authorizations
API key for authentication
Body
application/json
Response
Successful evaluation
Was this page helpful?
⌘I