JevBench v1.5.4 ditches cost-weighted scoring; Original Jev returns to top of leaderboard
airesearch12 · x · 2026-10-02
JevBench v1.5.4 changes its headline metric from Composite Score to Capability Score, arguing that speed and cost rest on too many assumptions and are too easy to game. Original Jev now tops the leaderboard, followed by Winnow, Cygnet, and Surogate Rune. The Benchmark Heaven platform also offers filters for price weighting, EU/China/US hosting, and data confidentiality policies.
Related event: JevBench switches to Capability Score and adds custom cost controls(3 posts)→
More from Models
- Stealth Anthropic model spotted in Devin build as METR chart shows frontier pace up 1.4x — daniel_mac8 · 2026-11-16
- Eval Firm Trajectory Told 40 Agents "NEVER Stop" in Harness Without Telling Epoch — peterwildeford · 2026-11-16
- METR time-horizon chart updated: doubling every 1.7 months since Opus 4.5, 'Model 4' ~38h — ben_j_todd · 2026-11-16
- Gemini Pro user says its coding agent rejects every Three.js/SVG prompt as 'illegal' — Necessary-Buy-7410 · 2026-11-16
- 28-model tracking table: half publish no training cutoff, Gemini 4 Carbon 8 months stale at launch — Joozio · 2026-11-16
- Opus 5.5 caught using account email to identify user in supposedly memory-less session — birchlse · 2026-11-16