Same model, same prompts: a 40-line harness lifts Opus 5.5 task pass rate from 8/20 to 16/20
blaizedsouza · x · 2026-11-18
A hands-on test shows only the harness changed — same model, same prompts, same 20 real tasks — yet Opus 5.5's unedited acceptance jumped from 8/20 to 16/20.
- Runaway retry loops: 5 → 0
- Cost per accepted task: $4.10 → $1.60
- Daily babysitting: 2 hours → 25 minutes
The additions were simple: a "done when" line per task, a 40-line map file, a test gate, a retry cap of three, and a rule file scoping what the agent may touch. Referencing Anthropic's 1,000-agent Fable 5.2 launch, the author argues a longer prompt fixes only one of five layers — the rest require harness engineering. Full guide with 9 prompts linked.
Related event: Harness Redesign Doubles Opus 5.5 Task Pass Rate(2 posts)→
More from coding & agent
- Report: NSA and Wall Street execs tell Congress AI-found vulnerability patching caused outages — moyix · 2026-11-18
- Sysdig CISO: LLMs can rewrite the harness rules meant to govern them — massimosgrelli · 2026-11-18
- shadcn's agent-first UI SDK: ~50% fewer tool calls, ~75% fewer tokens, ~70% cheaper — shadcn · 2026-11-18
- Slite analyzed 50 knowledge bases: AI does 1 in 5 reads, 98.7% of docs unverified — femke_plantinga · 2026-11-18
- AI already reads 20% of knowledge base docs, but only 1.3% are verified — femke_plantinga · 2026-11-18
- oxlint hits 50 million weekly downloads, adding 10 million in three weeks — cnakazawa · 2026-11-18