Ex-Chrome lead Addy Osmani: a decent model with a great harness beats a great model

aakashgupta · x · 2026-10-17

Aakash Gupta builds on Addy Osmani's claim that "a decent model with a great harness consistently beats a great model with a bad harness," laying out a 10-step roadmap for agent harness engineering with case studies from OpenAI, Anthropic, Ramp, Shopify, Amplitude, Notion and Apolio.

Definition: Agent = Model + Harness; same model jumped from 52.8 (outside top 30) to 66.5 (top 5) on Terminal-Bench 2.0 after harness changes.
Instructions: OpenAI compressed 1M lines into a 242-line AGENTS.md then a 110-line map, cutting tokens 23% with equal quality.
Context: Amplitude saw tool success rise from 9% to 76% with global context — richer context, not bigger models.
Skills: Apolio cut build-to-launch time 80–85% with sub-agents plus a skills library.
Memory: Anthropic-style fact.md pattern; one fact per file with reasoning.
Permissions: Ramp handles 65%+ of approvals via agents; hard limits enforced at every setting.
Tools: Shopify tiers by ambiguity (≤20 clear, 50+ need custom protocol); Notion shows 100+ tools only when needed.
Checks: Shopify's judge agreement with humans went from 0.02 to 0.69; failure rates under 5%/2%.
Loop: solo harness fixes a broken game in 20 min/$9; full harness 6 hrs/$200.
Start where mistakes are cheap, and put a hard limit — in code — on the action that would hurt most.

Original post →

Ex-Chrome lead Addy Osmani: a decent model with a great harness beats a great model

More from coding & agent