LangWatch
Sign Up

GoogleOpen weights model25.8B parameters

Gemma 4 26B A4B benchmarks

All models
  • General rank
    Too few tasks4 non-coding tasks ranked5 needed for a rank
  • Cost
    Self-hostedLarge GPU, ~61.9 GBNo hosted price
  • Traces#3 / 46 tied
    19.8%Best task · PII left invs PII detectors

Comparison summary

Strongest
Safety and guardrails
Leads 0 of 4 (0%)

Strongest at safety. It is ranked on 4 non-coding tasks, too few for a general rank (5 needed).

Specifications

Lab
Google
Weights
Open weights
Parameters
25.8B
Licence
Apache-2.0
Type
large LLM

Safety and guardrails

Share of tasks led · #7 of 46

  1. Gemma 4 31BLeads 3 of 4
  2. Perplexity PII TracerLeads 2 of 4
  3. Amazon ComprehendLeads 1 of 4
  4. Qwen3.8 27BLeads 1 of 5
  5. Gemma 4 12BLeads 0 of 4
  6. Ministral 3 14BLeads 0 of 5
  7. Gemma 4 26B A4B
    Leads 0 of 4
  8. GLiNER Multi PII v1Leads 0 of 4
See the 5 tasks

Traces

Tier 3 of 19#3 / 46

PII left in, % · Lower is better

  1. Amazon Comprehend10.6, interval 9.4 to 11.9, ahead alone
  2. GLiNER Multi PII v115.3, interval 13.2 to 17.3, tier 2
  3. NVIDIA GLiNER PII19.7, interval 17.4 to 22.0, tier 3
  4. Gemma 4 26B A4B
    19.8, interval 15.8 to 24.2, tier 3
  5. Qwen3.8 27B20.3, interval 16.5 to 24.3, tier 3
  6. GLiNER2 Large20.4, interval 18.0 to 22.8, tier 3
  7. Gemma 4 12B23.3, interval 19.0 to 27.6, tier 3
  8. Ministral 3 14B25.0, interval 20.6 to 29.3, tier 4
Tested against 52 PII detectors

Business

Tier 3 of 18#6 / 46

PII left in, % · Lower is better

  1. Perplexity PII Tracer2.1, interval 1.2 to 3.0, tied for the lead
  2. Qwen3.8 27B2.4, interval 1.7 to 3.3, tied for the lead
  3. Gemma 4 31B2.9, interval 1.9 to 4.0, tied for the lead
  4. GLiNER Multi PII v14.1, interval 3.1 to 5.3, tier 2
  5. Ministral 3 14B5.4, interval 4.0 to 6.9, tier 2
  6. Gemma 4 12B5.8, interval 4.4 to 7.3, tier 3
  7. Amazon Comprehend5.9, interval 4.6 to 7.2, tier 3
  8. Gemma 4 26B A4B
    6.9, interval 5.3 to 8.5, tier 3
Tested against 53 PII detectors

Multilingual

Tier 5 of 25#7 / 43

PII left in, % · Lower is better

  1. Perplexity PII Tracer2.9, interval 2.5 to 3.4, ahead alone
  2. Qwen3.8 27B4.2, interval 3.7 to 4.8, tier 2
  3. Gemma 4 31B4.5, interval 3.9 to 5.1, tier 2
  4. GLiNER Multi PII v15.7, interval 5.2 to 6.3, tier 3
  5. bardsai EU PII Multilang6.9, interval 6.3 to 7.5, tier 4
  6. Qwen3.6 35B A3B7.7, interval 7.0 to 8.4, tier 4
  7. Gemma 4 12B8.9, interval 8.1 to 9.7, tier 5
  8. Gemma 4 26B A4B
    9.3, interval 8.5 to 10.2, tier 5
Tested against 53 PII detectors

Look-alikes

Not ranked

Look-alikes redacted, % · Lower is better

  1. Gemma 4 26B A4B
    1.7, interval 0.8 to 2.9, not ranked
  2. Gemma 4 31B3.5, interval 2.1 to 5.1, ahead alone
  3. Qwen3.8 27B10.3, interval 7.9 to 12.9, tier 2
  4. NuExtract 319.8, interval 15.9 to 23.6, tier 3
  5. Qwen3.5 9B20.5, interval 16.8 to 24.3, tier 3
  6. Ministral 3 14B30.5, interval 26.5 to 34.8, tier 4
  7. NVIDIA GLiNER PII42.0, interval 38.2 to 46.1, tier 5
  8. Gemma 3 4B IT45.0, interval 41.0 to 49.0, tier 5
Tested against 53 PII detectors

LangWatch policy

Tier 4 of 13#5 / 42

PII left in, % · Lower is better

  1. Gemma 4 31B6.9, interval 2.8 to 11.3, ahead alone
  2. Ministral 3 14B10.2, interval 5.8 to 15.4, tier 2
  3. Gemma 4 12B16.5, interval 10.3 to 23.2, tier 3
  4. Qwen3.8 27B16.7, interval 9.8 to 25.0, tier 3
  5. Gemma 4 26B A4B
    20.5, interval 13.3 to 28.8, tier 4
  6. Qwen3.5 9B21.5, interval 13.4 to 30.6, tier 4
  7. Gemma 3 4B IT22.4, interval 14.3 to 32.1, tier 4
  8. Qwen3 4B Instruct 250729.6, interval 19.6 to 40.3, tier 5
Tested against 57 PII detectors

Cost

Self-hosted · GPU memory needed, among large LLMs · lower is better

  1. Ministral 3 14B~19 GB
  2. NuExtract 2.0 8B~19.9 GB
  3. Granite 4.2 8B~21.1 GB
  4. Qwen3.5 9B~23.2 GB
  5. Gemma 4 12B~28.7 GB
  6. Qwen3.6 35B A3B~45 GB
  7. Gemma 4 26B A4B
    ~61.9 GB
  8. Qwen3.8 27B~66.7 GB
How we count

A model leads a task when it is in the task's leading tie tier. Each task counts inside its own benchmark, against that benchmark's models; no score is averaged. An area chart shows the models ranked on at least 2 of the area's tasks in a benchmark this model is in too. The overall standing counts every non-coding tasks and needs 5 for a rank. Latency is compared only on one hardware tier.

Gemma 4 26B A4BOther modelsWhisker: 95% interval