Tuning your harness beats waiting 9 months: older model outperforms latest by ~10 points
VictorSanh · x · 2026-11-04
Victor Sanh (Hugging Face) pushes back on the 'we need the next model release' mindset at vertical AI companies: tuning an agent harness around a model 9 months older than the latest one beat the newest model by almost 10 points.
Evidence that the harness decides how much of the model you actually get:
- OpenAI tripled an ARC-AGI-3 score by changing two harness settings
- LangChain took Terminal Bench from 52.8 to 66.5 with the same model
- One tuning run brought GPT-5.2, GPT-5.4 Mini and GPT-5.6 Luna to 48%, past the latest model's 39%
Takeaway: before waiting nine months for the next release, tune the harness around what you already have.
More from coding & agent
- Karpathy's viral take: engineers' biggest regret is unsaved AI conversations — bfrench · 2026-11-04
- Apple lets apps bundle Siri skill files, devs mock the sloppy docs — msg · 2026-11-04
- YC-backed Simantic adds a simulation layer so AI agents can test firmware at software speed — ycombinator · 2026-11-04
- Bittensor Subnet 33 Launches Skill Store: Tested Skills Aim to Make Open Models Do Frontier Work — markjeffrey · 2026-11-04
- Building a 24-hour AI newsroom on Microsoft Copilot Harness: workflow beats prompts — WirelessLife · 2026-11-04
- Polyphonic to give AI agents their own Mac via open-source terminal-mac — RileyRalmuto · 2026-11-04