CloudflareOpen weights model27.4B parameters
Clef benchmarks
- General rankToo few tasks3 non-coding tasks ranked5 needed for a rank
- Speed#8 / 91,124 msMedian latency, p50Hosted API only
- Cost#11 / 11$3.22Per 1,000 decisionsHosted price
- Scenario judge#1 / 32 tied75.7%Best task · Accuracyvs Jev-class models
Comparison summary
- Strongest
- Judging and evals
- Leads 1 of 3 (33%)
A typed decision model: it answers a typed question, such as yes or no or one of a few options, with a probability for each answer and no text. Strongest at judging.
Strengths
- Catches the most failed simulated conversations when the criterion needs a calculation: about four in five
Weaknesses
- False alarms when judging real conversations: fails about one in three of the replies that are fine
- Misses made-up facts in real conversations: catches about two in five
Specifications
- Lab
- Cloudflare
- Weights
- Open weights
- Parameters
- 27.4B
- Licence
- Apache-2.0
Judging and evals
Share of tasks led · #3 of 32
- Mercury DecideLeads 2 of 3
- OpenAI Decisions APILeads 4 of 6
- ClefLeads 1 of 3
- deck-31BLeads 1 of 3
- pplx-decider v1.1 27BLeads 1 of 3
- GLiDELeads 1 of 3
- Gemini Decision 1Leads 1 of 3
- Quyet-1.0-LargeLeads 1 of 3
See the 3 tasks
LLM-as-a-judge
Tier 2 of 3#3 / 32Accuracy, % · Higher is better
- OpenAI Decisions API71.7, interval 66.1 to 77.2, tied for the lead
- Microsoft Decision 165.6, interval 59.0 to 72.5, tied for the lead
- Mercury Decide62.6, interval 56.6 to 69.1, tier 2
- Jev62.0, interval 56.1 to 68.3, tier 2
- Decision 2.0 Sol 2B61.3, interval 54.4 to 67.9, tier 2
- pplx-decider v1.1 27B61.3, interval 56.0 to 67.0, tier 2
- Quyet-1.0-Large61.3, interval 55.8 to 67.3, tier 2
- Clef58.3, interval 52.3 to 64.3, tier 2
Scenario judge
Tied for the lead#1 / 32Accuracy, % · Higher is better
- Mercury Decide83.3, interval 75.5 to 90.3, tied for the lead
- OpenAI Decisions API83.3, interval 76.1 to 89.9, tied for the lead
- deck-31B81.9, interval 74.8 to 88.1, tied for the lead
- Gemini Decision 180.6, interval 73.2 to 87.1, tied for the lead
- Quyet-1.0-Large79.2, interval 72.0 to 85.9, tied for the lead
- pplx-decider v1.1 27B76.4, interval 68.5 to 83.7, tied for the lead
- Clef75.7, interval 67.4 to 83.1, tied for the lead
- GLiDE75.0, interval 66.7 to 82.9, tier 2
Search
Tier 2 of 6#5 / 32Accuracy, % · Higher is better
- OpenAI Decisions API70.4, interval 66.0 to 74.8, tied for the lead
- Jev68.2, interval 63.0 to 73.1, tied for the lead
- Mercury Decide66.3, interval 61.5 to 71.0, tied for the lead
- GLiDE65.2, interval 60.9 to 69.4, tied for the lead
- Kev-27B61.1, interval 56.4 to 65.7, tier 2
- Clef60.2, interval 55.7 to 64.9, tier 2
- deck-31B58.9, interval 54.3 to 63.5, tier 2
- pplx-decider v1.1 27B58.9, interval 54.1 to 63.7, tier 2
Speed
Median latency, p50 · Hosted API only · lower is better
- OpenAI Decisions API225 ms
- Jev339 ms
- Gemini Decision 1370 ms
- Microsoft Decision 1390 ms
- Clef Omni455 ms
- Mercury Decide954 ms
- GLiDE961 ms
- Clef1,124 ms
Cost
Per 1,000 decisions, hosted price · lower is better
- pplx-decider v1.1 27B$0.064
- AutoJev-27B$0.16
- Clef Flash$0.21
- Microsoft Decision 1$0.25
- Jev$0.43
- Mercury Decide$0.52
- OpenAI Decisions API$1.23
- Clef$3.22
How we count
A model leads a task when it is in the task's leading tie tier. Each task counts inside its own benchmark, against that benchmark's models; no score is averaged. An area chart shows the models ranked on at least 2 of the area's tasks in a benchmark this model is in too. The overall standing counts every non-coding tasks and needs 5 for a rank. Latency is compared only on one hardware tier.
ClefOther modelsWhisker: 95% interval

