SlopCodeBench: Best AI Coding Agent Solved Just 14.8% of Evolving Task Checkpoints
aj_marshall · reddit · 2026-10-19
SlopCodeBench (arXiv 2603.24755, UW–Madison) tests what most benchmarks skip: how agent code degrades when requirements keep evolving. Agents extend their own prior code across checkpoints with no chat history — closer to real Claude Code usage.
Key findings:
- 15 models tested (Opus 4.5/4.6/4.7, Sonnet 4.6, etc.); none fully solved any of 36 problems. Best strict checkpoint solve rate: 14.8% (GPT 5.5); Opus 4.6 had the highest core test pass rate at 67.3%.
- Structural erosion increased in 77% of runs, verbosity in 75.5%; agent code is 2.3x more verbose than code from 473 human-maintained Python repos.
- In one case, Opus 4.7's main() grew from cyclomatic complexity 13 to 92 over 8 checkpoints (38 → 240 lines), worsening every step.
- "Anti-slop" and plan-first prompts cut initial verbosity/erosion by up to a third but didn't stop long-run degradation, and raised cost per checkpoint 12% (tested on GPT models only).
The author also released a video essay walkthrough.
More from Models
- MBZUAI Releases Jais 3: Six Open Arabic Models from 0.9B to 375B with Full Training Recipe — lmoroney · 2026-10-19
- Opus 5.5 hailed as a milestone: developers say they've stopped chasing new models — dreamwieber · 2026-10-19
- Six frontier AIs design egg-drop capsules; Claude Opus 5.5 survives 20 feet — 141_1337 · 2026-10-19
- Google AI Pro users from Pixel/Jio/student offers locked out of Gemini 4 Argon in Antigravity — MicahBerkley · 2026-10-19
- Gwern's "Truesight" Directory Collects LLM Stylometry De-anonymization Research — voooooogel · 2026-10-19
- Unverified leak claims a "Gemini 4 Carbon" model, backed only by a logo image — BusyNomad · 2026-10-19