JevBench to rotate hidden test sets as decider-4b v2 beats Jev on speed and cost
airesearch12 · x · 2026-09-25
The JevBench maintainer says the benchmark will evolve: hidden test sets will rotate to prevent benchmaxxing, and intelligence may get more weight in the next version. Currently decider-4b v2 trails Jev on intelligence (53.1 vs 49.4) and calibration but wins overall by being 5x faster and roughly half the price — JevBench weighs smart, calibrated, fast, and cheap equally, with a sort-by-intelligence option.
Related event: Open-source 4B model tops JevBench on speed and cost, not intelligence(3 posts)→
More from Models
- Gemini 4 Carbon matches predecessor's score but explores less, gets distracted — patience_cave · 2026-11-11
- Fable 5.2 takes MazeBench frontier, matches GPT-6 Astra puzzle-solving — patience_cave · 2026-11-11
- MazeBench: Fable 5.2 scores 15%, near-frontier at half price; Gemini 4 Carbon 11% — patience_cave · 2026-11-11
- ERNIE 5.5 Flash Preview enters LMArena Battle Mode for crowdsourced ranking — arena · 2026-11-11
- Indie dev flips from $20 Pro back to $200 Gemini Ultra after two days of deep use — yihui_indie · 2026-11-11
- Leak: Google's upcoming model lineup includes Hydrogen, Krypton, Helix and a video model — koltregaskes · 2026-11-11