29 Bugs in Benchmark History Have Never Been Fixed by Any Model
PawelHuryn · x · 2026-10-13
Pawel Huryn notes that across all runs in the benchmark's history, 29 bugs have never been fixed by anything. Models do fix bugs in real life, he adds, but usually only after he or users first detected them in the app — highlighting the gap between benchmark progress and real-world bug discovery and repair.
Related event: Pawel Huryn Benchmarks Muse Wonder 1.0: GPT-6 Astra Leads at 45 Points(5 posts)→
More from coding & agent
- AI creator reveals 5 unreleased Claude Skills, from video editing to personal agents — op7418 · 2026-10-13
- LangSmith launches AI Search: query agent traces in natural language — LangChain · 2026-10-13
- 30 AI-generated video clips turned into playable browser games with Claude — OdinLovis · 2026-10-13
- Dev builds a web design research skill that gets Claude to create bold branding concepts — round · 2026-10-13
- Kernel Panic conference in Madrid packs house, slides from four production-AI talks published — SergioPaniego · 2026-10-13
- This Hermes agent auto-filters weekly GitHub AI repos, search and fetch are free — prasenx · 2026-10-13