Reddit proposal: replace benchmark numbers with per-task red/yellow/green ratings
Toothpasteweiner · reddit · 2026-11-12
A Reddit user argues numeric benchmarks increasingly mislead users: aggregate scores get inflated by ever-harder tasks like 3D modeling and math proofs that don't reflect real work. He proposes granular poor/moderate/good per-task ratings — e.g. "Ctests: green, binary decompilation: yellow, fiction writing: red" — plus cost per task, which would help users pick the cheapest capable model, improve auto-routing, and serve local LLM users. He notes ArtificialAnalysis and HF leaderboards only show numbers and asks if such a tool exists.
More from Models
- Microsoft's MAI Transcribe 2 sees 12x usage growth, hitting 8.5M weekly transcriptions — mustafasuleyman · 2026-11-12
- Focus less on using the best model, more on using the model best — daniel_mac8 · 2026-11-12
- YC F26 Startup agenticdataco Launches Real-World Speech Data and Audio-Native RL Environments — ycombinator · 2026-11-12
- Microsoft's diagnostic AI solves 83% of NEJM cases vs 36% for physicians — PeterDiamandis · 2026-11-12
- Petition demands OpenAI permanently keep ChatGPT Standard Voice Mode and all 9 voices — shaunralston · 2026-11-12
- Researcher praising uncensored Kimi 3.1 Pro gets his X account hacked — erikphoel · 2026-11-12