JevBench v1.5.4 ditches cost-weighted scoring; Original Jev returns to top of leaderboard
airesearch12 · x · 2026-10-02
JevBench v1.5.4 changes its headline metric from Composite Score to Capability Score, arguing that speed and cost rest on too many assumptions and are too easy to game. Original Jev now tops the leaderboard, followed by Winnow, Cygnet, and Surogate Rune. The Benchmark Heaven platform also offers filters for price weighting, EU/China/US hosting, and data confidentiality policies.
More from Models
- JevBench to add evals for LLM routing, RAG retrieval, and moderation use cases — airesearch12 · 2026-10-02
- Google's Argon battle-tested by 200k+ Googlers daily, not benchmaxxed — Zergylord · 2026-10-02
- JEV claims to be the first System One model hosted in the EU — juanviera23 · 2026-10-02
- Gemini knew a user's mom's name unprompted, then gave three conflicting explanations — Exact_Firefighter864 · 2026-10-02
- GPT-6.1 Sol nearly matches Astra at a quarter of the price on RareBench — danielmckinn0n · 2026-10-02
- New results from PostTrainBench v1.2 are in — mariofilhoml · 2026-10-02