$266 experiment: messy code doesn't hurt Opus 5.5 across 196 SlopCodeBench checkpoints
aj_marshall · reddit · 2026-10-26
A developer tested SlopCodeBench's claim that agent code degrades and causes failures by running Opus 5.5 through all 196 checkpoints ($266 in API costs). Findings:
- Code did get messier: erosion and verbosity rose in 22 of 35 problems.
- But messiness didn't predict whether the next checkpoint passed within a problem.
- Only 0.3% of previously passing tests regressed; messier repos didn't regress more.
- Messiness didn't raise costs, but code size did: each doubling of the codebase added 19% to the next checkpoint's cost.
- Across the leaderboard, code cleanliness has near-zero correlation with problems solved.
The headline 17% strict solve rate undersells Opus — it passed 84-92% on new-spec tests. Observational, single-model, single-run caveats apply.
More from coding & agent
- VibeVideoBench: multimodal agents coding videos often fail on time management — Muxi Diao · 2026-10-28
- Dev builds open-source Excalidraw MCP after AI-generated diagrams kept overlapping text and arrows — pikahikmag · 2026-10-28
- DeepSeek vs Haiku pricing debunk: accounting for peak rates and token use, 1017 is 2-2.5x cheaper — teortaxesTex · 2026-10-28
- Viral screenshot shows an agent spawning 32 parallel workspaces in ~30 seconds — cephaloform · 2026-10-28
- Grok Bot 0.78.1 hides unreleased per-bot logins with manager-only team sharing — faraz0x · 2026-10-28
- Uber Built a Legal Redlining Agent for Its Lawyers—Here's What It Learned — AxSaucedo · 2026-10-28