33.3% of ImageNet images contain more than one class — the answer key itself is flawed
RexDouglass · x · 2026-09-03
Researcher klaaracz points out that ImageNet images often don't contain exactly one object: 33.3% show more than one ImageNet class (earlier reannotations estimated 14.9-21.6%), and 3.8% contain nothing from ImageNet at all. In other words, models have been graded against an incomplete answer key.
More from Models
- Muse 1.3 enters the frontier: investor says the race is now four-horse, not two — GavinSBaker · 2026-09-03
- Asking Fable 5.1 to Rewrite 'LLM Slop' Docs Immediately Trips Its 'General Harm' Safeguard — StewartalsopIII · 2026-09-03
- antirez Says His Latest Creation Stays on His Website, Unsure About Hugging Face Upload — antirez · 2026-09-03
- Why does Google AI give different answers to different people for the same query? — djnotskrillex · 2026-09-03
- Study finds all 13 major LLMs flip truth judgments on speaker gender, up to 23.6% of statements — anthara_ai · 2026-09-03
- Anthropic investigating elevated errors on Claude Sonnet 5, status page incident open — ClaudeAI-mod-bot · 2026-09-03