SlopCodeBench: no model solves all 36 problems, AI code 2.3x more verbose than human repos
aj_marshall · reddit · 2026-10-19
SlopCodeBench (UW–Madison, arXiv 2603.24755) tests what happens to agent code as requirements evolve across checkpoints, with no chat history — closer to real Claude Code usage. Of 15 models tested (Opus 4.5-4.7, Sonnet 4.6, GPT 5.5...), none fully solved any of 36 problems; best strict solve rate was 14.8% (GPT 5.5). Structural erosion appeared in 77% of runs, verbosity in 75.5%, and AI code was 2.3x more verbose than 473 human-maintained Python repos — e.g. Opus 4.7's main() grew from cyclomatic complexity 13 to 92 over 8 checkpoints. Anti-slop and plan-first prompts cut initial degradation by 1/3 but didn't stop long-term decay, while raising cost 12%.
Related event: SlopCodeBench: Top Coding Agents Struggle as Requirements Evolve(2 posts)→
More from coding & agent
- Redditor builds local "AgentOS" with approval gates, catches a job burning 197k tokens vs 29k — AnimeBrian · 2026-10-19
- Sound Lab by React Bits: 200 free UI/video sounds that install as code — GCWebDesigner · 2026-10-19
- A ~56KB meditative HTML page one-shot with Gemini using Antigravity — Saboo_Shubham_ · 2026-10-19
- How do you let your AI agents spend money? Dev surveys real-world payment controls — Twirledd · 2026-10-19
- JPMorgan, Citi, BofA block AI agents from accounts — approval gates are the architecture — ur_piyo_a_hoe · 2026-10-19
- Qwen Code v0.25.3 Ships Managed-Agent Fixes and Web-Shell Tweaks — qwen-code-review-bot · 2026-10-19