LangWatchOpen weights model29.4B parameters
Vega 27B Fine-Tuned (LangWatch) benchmarks
- CostSelf-hostedHardware not recordedNo hosted price
Comparison summary
A typed decision model: it answers a typed question, such as yes or no or one of a few options, with a probability for each answer and no text.
Specifications
- Lab
- LangWatch
- Weights
- Open weights
- Parameters
- 29.4B
- Licence
- Not recorded
Judging and evals
Tested on 3 tasks here, none ranked.
See the 3 tasks
LLM-as-a-judge
Not rankedAccuracy, % · Higher is better
- OpenAI Decisions API71.7, interval 66.1 to 77.2, tied for the lead
- Vega 27B Fine-Tuned (LangWatch)67.0, interval 59.9 to 74.5, not ranked
- Microsoft Decision 165.6, interval 59.0 to 72.5, tied for the lead
- Mercury Decide62.6, interval 56.6 to 69.1, tier 2
- Jev62.0, interval 56.1 to 68.3, tier 2
- Decision 2.0 Sol 2B61.3, interval 54.4 to 67.9, tier 2
- pplx-decider v1.1 27B61.3, interval 56.0 to 67.0, tier 2
- Quyet-1.0-Large61.3, interval 55.8 to 67.3, tier 2
Scenario judge
Not rankedAccuracy, % · Higher is better
- Mercury Decide83.3, interval 75.5 to 90.3, tied for the lead
- OpenAI Decisions API83.3, interval 76.1 to 89.9, tied for the lead
- deck-31B81.9, interval 74.8 to 88.1, tied for the lead
- Gemini Decision 180.6, interval 73.2 to 87.1, tied for the lead
- Quyet-1.0-Large79.2, interval 72.0 to 85.9, tied for the lead
- pplx-decider v1.1 27B76.4, interval 68.5 to 83.7, tied for the lead
- Clef75.7, interval 67.4 to 83.1, tied for the lead
- Vega 27B Fine-Tuned (LangWatch)74.3, interval 65.4 to 82.0, not ranked
Search
Not rankedAccuracy, % · Higher is better
- Vega 27B Fine-Tuned (LangWatch)75.6, interval 70.8 to 80.3, not ranked
- OpenAI Decisions API70.4, interval 66.0 to 74.8, tied for the lead
- Jev68.2, interval 63.0 to 73.1, tied for the lead
- Mercury Decide66.3, interval 61.5 to 71.0, tied for the lead
- GLiDE65.2, interval 60.9 to 69.4, tied for the lead
- Kev-27B61.1, interval 56.4 to 65.7, tier 2
- Clef60.2, interval 55.7 to 64.9, tier 2
- deck-31B58.9, interval 54.3 to 63.5, tier 2
How we count
A model leads a task when it is in the task's leading tie tier. Each task counts inside its own benchmark, against that benchmark's models; no score is averaged. An area chart shows the models ranked on at least 2 of the area's tasks in a benchmark this model is in too. The overall standing counts every non-coding tasks and needs 5 for a rank. Latency is compared only on one hardware tier.
Vega 27B Fine-Tuned (LangWatch)Other modelsWhisker: 95% interval

