Models
- JevTypeSafeProprietaryJev vs frontier LLMsJev vs open models up to 28BJev vs tiny open modelsLLM judgesJev vs Jev-class models
- Eikos-27BCaio VicentinoOpen weightsJev vs open models up to 28B
- AutoJev-27BCommunity (Hugging Face)Open weightsJev vs open models up to 28B
- Claude Opus 5.5AnthropicProprietaryJev vs frontier LLMsLLM judges
- Claude Sonnet 5.5AnthropicProprietaryJev vs frontier LLMsLLM judges
- Claude Haiku 4.5AnthropicProprietaryJev vs frontier LLMs
- GPT-6.1 SolOpenAIProprietaryJev vs frontier LLMsLLM judges
- GPT-6 LunaOpenAIProprietaryJev vs frontier LLMs
- GPT-5.6 TerraOpenAIProprietaryJev vs frontier LLMs
- Gemini 3.8 FlashGoogleProprietaryJev vs frontier LLMsLLM judges
- Gemma 4 31BGoogleOpen weightsPII detection
- Gemma 4 26B A4BGoogleOpen weightsPII detection
- Gemma 4 12BGoogleOpen weightsPII detection
- Qwen3.8 27BQwenOpen weightsPII detection
- Qwen3.6 35B A3BQwenOpen weightsPII detection
- Qwen3.5 9BQwenOpen weightsPII detection
- Llama 3.2 3B InstructMetaOpen weightsPII detection
- Ministral 3 14BMistral AIOpen weightsPII detection
- Granite 4.2 8BIBMOpen weightsPII detection
- Phi-4 Mini InstructMicrosoftOpen weightsPII detection
General ranking
| # | Model | Tasks led | Counted in | Median latency | Per 1k decisions |
|---|---|---|---|---|---|
| 1 | Claude Opus 5.52,194 ms · $3.44AnthropicFrontier LLMs 8/10LLM judges 3/3 | 11 of 13 (85%) | Frontier LLMs 8/10LLM judges 3/3 | 2,194 msHosted API | $3.44Hosted price |
| 2 | Jev~65 ms · $0.029TypeSafeFrontier LLMs 2/10Open models to 28B 11/14Tiny open models 14/14LLM judges 0/3Jev-class models 3/3 | 30 of 44 (68%) | Frontier LLMs 2/10Open models to 28B 11/14Tiny open models 14/14LLM judges 0/3Jev-class models 3/3 | ~65 msHosted API | $0.029Hosted price |
| 3 | AutoJev-27B99 ms · Self-hostedCommunity (Hugging Face)Open models to 28B 8/12 | 8 of 12 (67%) | Open models to 28B 8/12 | 99 msH100 | Self-hostedRan on H100 |
| 4 | Eikos-27B82 ms · Self-hostedCaio VicentinoOpen models to 28B 6/10 | 6 of 10 (60%) | Open models to 28B 6/10 | 82 msH100 | Self-hostedRan on H100 |
| 5 | GPT-6.1 SolNot measured · ~$1.02OpenAIFrontier LLMs 5/10LLM judges 2/3 | 7 of 13 (54%) | Frontier LLMs 5/10LLM judges 2/3 | Not measured | ~$1.02Estimated price |
| 6 | Claude Sonnet 5.51,646 ms · $1.52AnthropicFrontier LLMs 5/10LLM judges 1/3 | 6 of 13 (46%) | Frontier LLMs 5/10LLM judges 1/3 | 1,646 msHosted API | $1.52Hosted price |
| 7 | Shisa DE-146 ms · Self-hostedshisa-aiOpen models to 28B 5/12 | 5 of 12 (42%) | Open models to 28B 5/12 | 46 msH100 | Self-hostedRan on H100 |
| 8 | Gemini 3.8 Flash1,936 ms · $0.57GoogleFrontier LLMs 4/10LLM judges 1/3 | 5 of 13 (38%) | Frontier LLMs 4/10LLM judges 1/3 | 1,936 msHosted API | $0.57Hosted price |
| 9 | GPT-5.6 Terra1,565 ms · $1.03OpenAIFrontier LLMs 3/10 | 3 of 10 (30%) | Frontier LLMs 3/10 | 1,565 msHosted API | $1.03Hosted price |
| 10 | Qwen3.8 27BNot measured · Self-hostedQwenPII detection 1/5 | 1 of 5 (20%) | PII detection 1/5 | Not measured | Self-hostedLarge GPU, ~66.7 GB |
| 11 | GPT-6 Luna1,124 ms · $0.049OpenAIFrontier LLMs 2/10 | 2 of 10 (20%) | Frontier LLMs 2/10 | 1,124 msHosted API | $0.049Hosted price |
| 12 | Claude Haiku 4.51,306 ms · $0.69AnthropicFrontier LLMs 1/10 | 1 of 10 (10%) | Frontier LLMs 1/10 | 1,306 msHosted API | $0.69Hosted price |
| 13 | SemIf (Qwen3.5-4B)108 ms · Self-hostedQwenOpen models to 28B 1/11 | 1 of 11 (9%) | Open models to 28B 1/11 | 108 msL4 | Self-hostedRan on L4 |
| 14 | Kev-9B43 ms · Self-hostedjaredpalmerOpen models to 28B 0/8 | 0 of 8 (0%) | Open models to 28B 0/8 | 43 msH100 | Self-hostedRan on H100 |
| 15 | Eikos-4B95 ms · Self-hostedCaio VicentinoOpen models to 28B 0/9 | 0 of 9 (0%) | Open models to 28B 0/9 | 95 msL4 | Self-hostedRan on L4 |
| 16 | Ministral 3 14BNot measured · Self-hostedMistral AIPII detection 0/5 | 0 of 5 (0%) | PII detection 0/5 | Not measured | Self-hostedMid GPU, ~19 GB |
| 17 | Kev-4B85 ms · Self-hostedjaredpalmerOpen models to 28B 0/12 | 0 of 12 (0%) | Open models to 28B 0/12 | 85 msL4 | Self-hostedRan on L4 |
| 18 | Kev-0.8B85 ms · Self-hostedjaredpalmerOpen models to 28B 0/12Tiny open models 0/12 | 0 of 24 (0%) | Open models to 28B 0/12Tiny open models 0/12 | 85 msLaptop GPU | Self-hostedRan on Laptop GPU |
| 19 | Laya-typed53 ms · Self-hostedconvaiinnovationsOpen models to 28B 0/13Tiny open models 0/13 | 0 of 26 (0%) | Open models to 28B 0/13Tiny open models 0/13 | 53 msLaptop GPU | Self-hostedRan on Laptop GPU |
| 20 | Kev-0.6B61 ms · Self-hostedjaredpalmerOpen models to 28B 0/12Tiny open models 0/12 | 0 of 24 (0%) | Open models to 28B 0/12Tiny open models 0/12 | 61 msLaptop GPU | Self-hostedRan on Laptop GPU |
| 21 | Laya50 ms · Self-hostedconvaiinnovationsOpen models to 28B 0/13Tiny open models 0/13 | 0 of 26 (0%) | Open models to 28B 0/13Tiny open models 0/13 | 50 msLaptop GPU | Self-hostedRan on Laptop GPU |
| 22 | SimpleJev (Qwen3.5-0.8B)196 ms · Self-hostedQwenOpen models to 28B 0/7Tiny open models 0/7 | 0 of 14 (0%) | Open models to 28B 0/7Tiny open models 0/7 | 196 msLaptop GPU | Self-hostedRan on Laptop GPU |
| 23 | Qwen3.5 9BNot measured · Self-hostedQwenPII detection 0/5 | 0 of 5 (0%) | PII detection 0/5 | Not measured | Self-hostedMid GPU, ~23.2 GB |
| 24 | GLiNER2.5-Decide61 ms · Self-hostedFastinoOpen models to 28B 0/12 | 0 of 12 (0%) | Open models to 28B 0/12 | 61 msLaptop GPU | Self-hostedRan on Laptop GPU |
| 25 | SemIf (Qwen3-0.6B)60 ms · Self-hostedQwenOpen models to 28B 0/7Tiny open models 0/7 | 0 of 14 (0%) | Open models to 28B 0/7Tiny open models 0/7 | 60 msLaptop GPU | Self-hostedRan on Laptop GPU |
| 26 | openJev Verdict 1.445 ms · Self-hostedheman10xOpen models to 28B 0/5Tiny open models 0/5 | 0 of 10 (0%) | Open models to 28B 0/5Tiny open models 0/5 | 45 msLaptop GPU | Self-hostedRan on Laptop GPU |
| 27 | Gemma 3 4B ITNot measured · Self-hostedGooglePII detection 0/5 | 0 of 5 (0%) | PII detection 0/5 | Not measured | Self-hostedSmall GPU, ~3.1 GB |
| Too few tasks to rank (under 5) | |||||
| Gemma 4 12BNot measured · Self-hostedGooglePII detection 0/4 | 0 of 4 (0%) | PII detection 0/4 | Not measured | Self-hostedLarge GPU, ~28.7 GB | |
| Gemma 4 26B A4BNot measured · Self-hostedGooglePII detection 0/4 | 0 of 4 (0%) | PII detection 0/4 | Not measured | Self-hostedLarge GPU, ~61.9 GB | |
| Gemma 4 31BNot measured · Self-hostedGooglePII detection 3/4 | 3 of 4 (75%) | PII detection 3/4 | Not measured | Self-hostedLarge GPU, ~75 GB | |
| Granite 4.2 8BNot measured · Self-hostedIBMPII detection 0/4 | 0 of 4 (0%) | PII detection 0/4 | Not measured | Self-hostedMid GPU, ~21.1 GB | |
| Qwen3.6 35B A3BNot measured · Self-hostedQwenPII detection 0/4 | 0 of 4 (0%) | PII detection 0/4 | Not measured | Self-hostedLarge GPU, ~45 GB | |
| Phi-4 Mini InstructNot measured · Self-hostedMicrosoftPII detection 0/3 | 0 of 3 (0%) | PII detection 0/3 | Not measured | Self-hostedSmall GPU, ~3.1 GB | |
| Llama 3.2 3B InstructNot measured · Self-hostedMeta | No ranked task | Not measured | Self-hostedSmall GPU, ~2.6 GB | ||
Share of tasks in the leading tie tier, not an averaged score. Each task counts inside its own benchmark, against that benchmark's models. Ties go to the lower average rank; under 5 ranked tasks, no rank. Coding tasks (commit types, code and logs) are left out. Latency is compared only within one hardware tier; ~ marks an estimate.
All models
- Shisa DE-1shisa-aiLeads 5 of 12
- SemIf (Qwen3.5-4B)QwenLeads 1 of 11
- Kev-9BjaredpalmerLeads 0 of 8
- Eikos-4BCaio VicentinoLeads 0 of 9
- Kev-4BjaredpalmerLeads 0 of 12
- Kev-0.8BjaredpalmerLeads 0 of 24
- Laya-typedconvaiinnovationsLeads 0 of 26
- Kev-0.6BjaredpalmerLeads 0 of 24
- LayaconvaiinnovationsLeads 0 of 26
- SimpleJev (Qwen3.5-0.8B)QwenLeads 0 of 14
- GLiNER2.5-DecideFastinoLeads 0 of 12
- SemIf (Qwen3-0.6B)QwenLeads 0 of 14
- openJev Verdict 1.4heman10xLeads 0 of 10
- Gemma 3 4B ITGoogleLeads 0 of 5
- Amazon ComprehendAmazonLeads 1 of 4
- bardsai EU PII MultilangbardsaiLeads 0 of 4
- BERT base NER (dslim)dslimLeads 0 of 4
- BetterLeaksNot recordedLeads 0 of 4
- CredSweeperSamsungLeads 0 of 4
- DataFogDataFogLeads 0 of 4
- detect-secretsYelpLeads 0 of 4
- GitleaksNot recordedLeads 0 of 4
- GLiNER Multi PII v1urchadeLeads 0 of 4
- GLiNER2 LargeFastinoLeads 0 of 4
- KingfisherMongoDBLeads 0 of 4
- Knowledgator GLiNER PII LargeKnowledgatorLeads 0 of 4
- NuExtract 2.0 8BNuMindLeads 0 of 4
- NuExtract 3NuMindLeads 0 of 4
- NVIDIA GLiNER PIINVIDIALeads 0 of 4
- OpenMed Privacy Filter MultilingualOpenMedLeads 0 of 4
- Perplexity PII TracerPerplexityLeads 2 of 4
- phonenumbersNot recordedLeads 0 of 4
- Presidio (all recognizers)MicrosoftLeads 0 of 4
- Presidio (default)MicrosoftLeads 0 of 4
- Presidio (LangWatch config)MicrosoftLeads 0 of 4
- Qwen3 4B Instruct 2507QwenLeads 0 of 4
- Secret Masker v3.3aandrewandrewsenLeads 0 of 4
- secretlintNot recordedLeads 0 of 4
- TitusNot recordedLeads 0 of 4
- TruffleHogTruffle SecurityLeads 0 of 4
- Veil PII KO Lite1tLeads 0 of 4
- XLM-RoBERTa NER HRLDavlanLeads 0 of 4
- Anonym-IA CamemBERT PII FRAnonym-IALeads 0 of 3
- ClefCloudflareLeads 3 of 3
- Clef FlashCloudflareLeads 0 of 3
- CygnetblockbrainLeads 0 of 3
- decider-4b v2MapikaLeads 0 of 3
- Decision 2.0 Eos 0.8BvLLM Semantic RouterLeads 0 of 3
- Decision 2.0 Kai 0.6BvLLM Semantic RouterLeads 0 of 3
- Decision 2.0 Lux 9BvLLM Semantic RouterLeads 0 of 3
- Decision 2.0 Nox 4BvLLM Semantic RouterLeads 0 of 3
- Decision 2.0 Sol 2BvLLM Semantic RouterLeads 0 of 3
- Decision 2.0 Vega 27BvLLM Semantic RouterLeads 3 of 3
- DiffusionGemma-JevGoogleLeads 1 of 3
- GLiDEFastinoLeads 3 of 3
- Granite 4.0 MicroIBMLeads 0 of 3
- Hopper 12BHopitAILeads 1 of 3
- Jev-Omniakhilaaa3Leads 3 of 3
- JevK5alibiserikbayLeads 0 of 3
- Julia-1Supersonic LabsLeads 0 of 3
- Kev-27BjaredpalmerLeads 3 of 3
- Lev 4Binterfaze-aiLeads 0 of 3
- OpenAI Privacy FilterOpenAILeads 0 of 3
- OpenMed QwenMed XLOpenMedLeads 0 of 3
- Qwen3 1.7BQwenLeads 0 of 3
- spaCy en_core_web_lgExplosionLeads 0 of 3
- WinnowEldanRingLeads 3 of 3
- Gravitee BERT Small PIIGraviteeLeads 0 of 2
- spaCy de_core_news_lgExplosionLeads 0 of 2
- Gemma 3 1B ITGoogleLeads 0 of 1
- LFM2 1.2BLiquid AILeads 0 of 1
- LFM2.5 1.2B InstructLiquid AILeads 0 of 1
- Llama 3.2 1B InstructMetaLeads 0 of 1
- Ministral 3 3B InstructMistral AILeads 0 of 1
- NuExtract 2.0 2BNuMindLeads 0 of 1
- Qwen3 0.6BQwenLeads 0 of 1
- Qwen3.5 2BQwenLeads 0 of 1
- SmolLM3 3BHugging FaceLeads 0 of 1
- GDPR Anonymization 0.5BNot recordedNot ranked
- Qwen3.5 0.8BQwenNot ranked
