JevBench v1.4 adds 308 evolving sealed tasks to stop benchmaxxing, now covers 70+ models
airesearch12 · x · 2026-09-23
JevBench has released v1.4, now covering 70+ models with the top 5 being Jev, JevK5, Hopper, Winnow-12B Q8, and reflex 4B. The update focuses on anti-benchmaxxing:
- Evolving sealed test set: 308 new hard tasks that keep evolving, supplying 20% of the Intelligence score, to prevent public-set saturation and training-data leakage.
- Calibration: sealed results are blended into calibration at a moderated weight; a k=1 penalty targets public-to-sealed gaps above 25 pp, rewarding genuine generalization.
- Equal-weight harmonic mean across four axes.
- Speed and cost gating: brilliant models still need Jev-class speed and cost, or they get gated down.
- API transparency: the leaderboard now flags whose API endpoints received held-out test item texts; answers are never shared.
Related event: JevBench v1.4 Adds Over 70 Models with Anti-Gaming Safeguards(3 posts)→
More from Models
- Opus 5.5 feels nerfed — user blames system prompt restrictions, not the model — oyacaro · 2026-11-12
- ChatGPT now links to brand homepages far more, reshaping brand-mention tracking — lilyraynyc · 2026-11-12
- YC-backed Monte launches post-training platform to build proprietary models from your agents' real work — ycombinator · 2026-11-12
- Leak Claims OpenAI Will Launch Entire GPT-6.2 Family Tomorrow to Counter Opus 5.5 and Fable 5.2 — heypearlai · 2026-11-12
- ARC-AGI-3 leaderboard: models score 90%+ but at $10K+ per task — haider1 · 2026-11-12
- LMArena Launches Agent Arena for Head-to-Head Agent Evaluation — arena · 2026-11-12