AECI scores reported to two decimals, but the 5th significant digit is likely noise
dfrsrchtwts · x · 2026-09-23
A commenter argues that AECI benchmark scores reported to two decimal places (5 significant digits) imply a precision that isn't real: since scores could be shifted by a constant without changing meaning, even the 4th significant digit is probably meaningless — a critique of over-precise benchmark reporting.
Related event: AECI Benchmark Scores Criticized for False Precision(2 posts)→
More from Models
- OpenAI ships GPT-6 Sol and Luna at ~50% below GPT-5.6 pricing, with 90% cheaper cached reads — etherd0t · 2026-09-23
- Matt Shumer launches first Opus 5.5 Gauntlet Loop, live-testing the model to redesign his site — mattshumer_ · 2026-09-23
- Codex users say weekly limits burned in 2 days, questioning OpenAI's '50% cheaper' Sol/Luna pricing — cocktailpeanut · 2026-09-23
- Two studies: AI chatbots get financial advice wrong 57%-85% of the time — PtrPomorski · 2026-09-23
- Early GPT-6 Sol access: 1-hour tasks done in 20 minutes with half the tokens — flavioAd · 2026-09-23
- GPT-6 Sol lands in Perplexity as Computer's default Light option — perplexity_ai · 2026-09-23