GoogleProprietary model

Gemini 4 Hydrogen benchmarks

All models
  • General rank
    Too few tasks3 non-coding tasks ranked5 needed for a rank
  • Speed#6 / 7
    6,516 msMedian latency, p50Hosted API only
  • Cost#8 / 10
    $31Per 1,000 decisionsHosted price
  • Scenario judge#1 / 10 tied
    86.1%Best task · Accuracyvs frontier LLMs

Comparison summary

Strongest
Judging and evals
Leads 2 of 3 (67%)

A language model (LLM): it reads a prompt and writes text back, so the prompt sets the task. Strong at judging.

Specifications

Lab
Google
Weights
Proprietary
Parameters
Not disclosed
Licence
Proprietary

Judging and evals

Set

Share of tasks led · #2 of 10

  1. Claude Opus 5.5Leads 3 of 3
  2. Gemini 4 Hydrogen
    Leads 2 of 3
  3. GPT-6.1 SolLeads 2 of 3
  4. OpenAI Decisions APILeads 4 of 6
  5. Gemini 4 ArgonLeads 1 of 3
  6. Claude Sonnet 5.5Leads 1 of 3
  7. Gemini 3.8 FlashLeads 1 of 3
  8. JevLeads 1 of 6
See the 3 tasks

LLM-as-a-judge

Tier 2 of 5#3 / 10

Accuracy, % · Higher is better

  1. GPT-6.1 Sol92.6, interval 89.1 to 95.8, tied for the lead
  2. Claude Opus 5.589.1, interval 83.9 to 93.9, tied for the lead
  3. Gemini 4 Hydrogen
    87.0, interval 83.2 to 90.6, tier 2
  4. Gemini 4 Argon84.4, interval 79.7 to 89.1, tier 2
  5. Claude Sonnet 5.583.5, interval 78.4 to 88.5, tier 2
  6. GPT-6 Luna82.6, interval 77.5 to 87.8, tier 2
  7. Gemini 3.8 Flash80.9, interval 76.2 to 85.5, tier 3
  8. Claude Haiku 5.573.9, interval 67.9 to 80.3, tier 3

Scenario judge

Tied for the lead#1 / 10

Accuracy, % · Higher is better

  1. Claude Opus 5.586.8, interval 80.2 to 92.3, tied for the lead
  2. Gemini 4 Hydrogen
    86.1, interval 79.8 to 91.6, tied for the lead
  3. Gemini 4 Argon84.0, interval 77.7 to 89.6, tied for the lead
  4. Gemini 3.8 Flash83.3, interval 76.8 to 89.1, tied for the lead
  5. OpenAI Decisions API83.3, interval 76.1 to 89.9, tied for the lead
  6. Claude Sonnet 5.581.3, interval 74.1 to 87.4, tier 2
  7. GPT-6.1 Sol81.3, interval 74.1 to 87.3, tier 2
  8. GPT-6 Luna79.9, interval 72.8 to 86.2, tier 2

Search

Tied for the lead#1 / 10

Accuracy, % · Higher is better

  1. Claude Opus 5.591.5, interval 87.6 to 94.8, tied for the lead
  2. Gemini 4 Hydrogen
    90.0, interval 86.9 to 92.8, tied for the lead
  3. GPT-6.1 Sol89.6, interval 85.6 to 93.1, tied for the lead
  4. Claude Sonnet 5.589.1, interval 85.8 to 92.1, tied for the lead
  5. Gemini 3.8 Flash85.0, interval 81.3 to 88.4, tier 2
  6. Gemini 4 Argon81.7, interval 77.9 to 85.4, tier 2
  7. Claude Haiku 5.581.3, interval 76.9 to 85.4, tier 2
  8. GPT-6 Luna75.2, interval 70.7 to 79.6, tier 3

Speed

Median latency, p50 · Hosted API only · lower is better

  1. OpenAI Decisions API226 ms
  2. Jev318 ms
  3. Claude Haiku 5.51,966 ms
  4. GPT-6 Luna3,085 ms
  5. Gemini 3.8 Flash5,493 ms
  6. Gemini 4 Hydrogen
    6,516 ms
  7. Gemini 4 Argon6,714 ms

Cost

Per 1,000 decisions, hosted price · lower is better

How we count

A model leads a task when it is in the task's leading tie tier. Each task counts inside its own benchmark, against that benchmark's models; no score is averaged. An area chart shows the models ranked on at least 2 of the area's tasks in a benchmark this model is in too. The overall standing counts every non-coding tasks and needs 5 for a rank. Latency is compared only on one hardware tier.

Gemini 4 HydrogenOther modelsWhisker: 95% interval