MapikaOpen weights model4.2B parameters
decider-4b v2 benchmarks
- General rankToo few tasks3 non-coding tasks ranked5 needed for a rank
- Speed#4 / 8409 msMedian latency, p50L4 only
- CostSelf-hostedRan on L4No hosted price
- LLM-as-a-judge#3 / 32 tied53.0%Best task · Accuracyvs Jev-class models
Comparison summary
- Strongest
- Judging and evals
- Leads 0 of 3 (0%)
A typed decision model: it answers a typed question, such as yes or no or one of a few options, with a probability for each answer and no text. Strongest at judging.
Specifications
- Lab
- Mapika
- Weights
- Open weights
- Parameters
- 4.2B
- Licence
- Apache-2.0
Judging and evals
Share of tasks led · #19 of 32
- Mercury DecideLeads 2 of 3
- OpenAI Decisions APILeads 4 of 6
- ClefLeads 1 of 3
- deck-31BLeads 1 of 3
- pplx-decider v1.1 27BLeads 1 of 3
- GLiDELeads 1 of 3
- Gemini Decision 1Leads 1 of 3
- decider-4b v2Leads 0 of 3
See the 3 tasks
LLM-as-a-judge
Tier 2 of 3#3 / 32Accuracy, % · Higher is better
- OpenAI Decisions API71.7, interval 66.1 to 77.2, tied for the lead
- Microsoft Decision 165.6, interval 59.0 to 72.5, tied for the lead
- Mercury Decide62.6, interval 56.6 to 69.1, tier 2
- Jev62.0, interval 56.1 to 68.3, tier 2
- Decision 2.0 Sol 2B61.3, interval 54.4 to 67.9, tier 2
- pplx-decider v1.1 27B61.3, interval 56.0 to 67.0, tier 2
- Quyet-1.0-Large61.3, interval 55.8 to 67.3, tier 2
- decider-4b v253.0, interval 46.2 to 60.1, tier 2
Scenario judge
Tier 3 of 3#20 / 32Accuracy, % · Higher is better
- Mercury Decide83.3, interval 75.5 to 90.3, tied for the lead
- OpenAI Decisions API83.3, interval 76.1 to 89.9, tied for the lead
- deck-31B81.9, interval 74.8 to 88.1, tied for the lead
- Gemini Decision 180.6, interval 73.2 to 87.1, tied for the lead
- Quyet-1.0-Large79.2, interval 72.0 to 85.9, tied for the lead
- pplx-decider v1.1 27B76.4, interval 68.5 to 83.7, tied for the lead
- Clef75.7, interval 67.4 to 83.1, tied for the lead
- decider-4b v261.1, interval 51.3 to 70.6, tier 3
Search
Tier 3 of 6#11 / 32Accuracy, % · Higher is better
- OpenAI Decisions API70.4, interval 66.0 to 74.8, tied for the lead
- Jev68.2, interval 63.0 to 73.1, tied for the lead
- Mercury Decide66.3, interval 61.5 to 71.0, tied for the lead
- GLiDE65.2, interval 60.9 to 69.4, tied for the lead
- Kev-27B61.1, interval 56.4 to 65.7, tier 2
- Clef60.2, interval 55.7 to 64.9, tier 2
- deck-31B58.9, interval 54.3 to 63.5, tier 2
- decider-4b v245.0, interval 39.2 to 51.0, tier 3
Speed
Median latency, p50 · L4 only · lower is better
- Decision 2.0 Kai 0.6B114 ms
- Decision 2.0 Eos 0.8B250 ms
- Decision 2.0 Sol 2B269 ms
- decider-4b v2409 ms
- Lev 4B620 ms
- JevK5665 ms
- Decision 2.0 Nox 4B691 ms
- Winnow1,356 ms
How we count
A model leads a task when it is in the task's leading tie tier. Each task counts inside its own benchmark, against that benchmark's models; no score is averaged. An area chart shows the models ranked on at least 2 of the area's tasks in a benchmark this model is in too. The overall standing counts every non-coding tasks and needs 5 for a rank. Latency is compared only on one hardware tier.
decider-4b v2Other modelsWhisker: 95% interval

