jev Decision Index v0.3 adds a private benchmark half and vision tasks to curb benchmark gaming
maximelabonne · x · 2026-10-07
- jev Decision Index v0.3 ships with a much stronger benchmark set, including a private half designed so rankings track skills beyond public benchmarks.
- Vision tasks are added in this version, as promised. Spread by multimodalart.
Related event: SWE Decision Index v0.3 Adds Private Benchmarks and Vision to Curb Gaming(4 posts)→
More from Models
- Token-based pricing is strange: unpredictable, decoupled from value, misaligned incentives — amankhan · 2026-10-07
- JevBench splits leaderboard: open-weight models and API providers now ranked separately — airesearch12 · 2026-10-07
- Hot take: active params barely matter for cyber capability evals — RL env coverage is key — teortaxesTex · 2026-10-07
- Decagon launches Voice 3 with Chord voice model and duplex architecture for customer agents — Scobleizer · 2026-10-07
- JEV-9B, a Qwen3.5-based calibrated decision model, trends on Hugging Face — autotrust · 2026-10-07
- "Astra Pause Syndrome": steering may be making models go silent, OpenAI has a workaround — thursdai_pod · 2026-10-07