JevBench adds configurable cost tolerance: different budgets, different winners
airesearch12 · x · 2026-10-02
Benchmark Heaven's JevBench added a new feature letting users configure their own speed and cost boundaries for "Jev-class" models. The author notes that if 3.2x Jev cost is acceptable to you, the winner changes even on capability score.
Default settings remain max 2x cost and max 2x slowness — beyond that, a model can't be called an official JevBench ranking. The benchmark also supports filtering by price basis, hosting region (China/EU/US), and data confidentiality policies, highlighting the gap between cost-insensitive and cost-sensitive evaluation.
Related event: JevBench switches to Capability Score and adds custom cost controls(3 posts)→
More from Models
- AI's clever quantization move is just sensitivity analysis, an old pruning trick — code_star · 2026-11-16
- AI progress doubled since GTA VI's first trailer; GPT-4 Turbo now ranks 379th — ArtificialAnlys · 2026-11-16
- Tencent's Hy4 Flash builds a 3D game with shield mechanics in 63 minutes, beating Hy4's 90 — ShunyuYao12 · 2026-11-16
- Hy4 Flash 3D voxel demo outpaces DeepSeek-V4.1 in same-prompt comparison — ShunyuYao12 · 2026-11-16
- Yacine MTB claims training LLMs is 'easy', says he's shocked by how simple it is — yacineMTB · 2026-11-16
- MiniMax M3.1 Flash lands on OpenRouter, with Inferact as first provider — Xianbao_QIAN · 2026-11-16