JevBench to rotate hidden test sets as decider-4b v2 beats Jev on speed and cost
airesearch12 · x · 2026-09-25
The JevBench maintainer says the benchmark will evolve: hidden test sets will rotate to prevent benchmaxxing, and intelligence may get more weight in the next version. Currently decider-4b v2 trails Jev on intelligence (53.1 vs 49.4) and calibration but wins overall by being 5x faster and roughly half the price — JevBench weighs smart, calibrated, fast, and cheap equally, with a sort-by-intelligence option.
Related event: Open-source 4B model tops JevBench on speed and cost, not intelligence(3 posts)→
More from Models
- Hesamation jokes he's back in a 'toxic relationship' to try Claude Opus 5.5 — addyosmani · 2026-09-25
- Ethan Mollick: GPT-6 Astra beats Nethack on just its 3rd try — emollick · 2026-09-25
- AI models ran a vending machine business for a year: GPT-6 Sol turned $500 into $14,428 — 141_1337 · 2026-09-25
- Kevin Roose hands an AI agent $100 and a Kalshi account to test frontier models — MickeySteamboat · 2026-09-25
- Ex-Meta engineer benchmarks Jev vs GPT-nano: same score, 5x faster, 20% pricier — danielmckinn0n · 2026-09-25
- Altimeter CEO: OpenAI's Navier–Stokes-solving model withheld amid safety and govt scrutiny — rohanpaul_ai · 2026-09-25