JevBench adds configurable cost tolerance: different budgets, different winners
airesearch12 · x · 2026-10-02
Benchmark Heaven's JevBench added a new feature letting users configure their own speed and cost boundaries for "Jev-class" models. The author notes that if 3.2x Jev cost is acceptable to you, the winner changes even on capability score.
Default settings remain max 2x cost and max 2x slowness — beyond that, a model can't be called an official JevBench ranking. The benchmark also supports filtering by price basis, hosting region (China/EU/US), and data confidentiality policies, highlighting the gap between cost-insensitive and cost-sensitive evaluation.
Related event: JevBench Switches to Capability Score and Adds Custom Cost Thresholds(2 posts)→
More from Models
- Users report queries being routed to rumored Fable 5.5 as launch nears — kimmonismus · 2026-10-02
- OpenAI GPT-6.1 Sol and Gemini 4 Argon both launch at identical $2/$10 pricing — AccBalanced · 2026-10-02
- Arena now runs up to 600,000 E2B sandboxes a day for model evaluation — badphilosopher · 2026-10-02
- OpenAI's $500 plan now offers less usage than its old $100 tier, users fume — petrusenko_max · 2026-10-02
- Grok is now live inside XChat group chats for US Premium+ subscribers — XFreeze · 2026-10-02
- Burkov: on truly novel problems, LLM error rates tell you how often you'll be fooled — burkov · 2026-10-02