Strong Models Design Harnesses for Weak Ones: Accuracy Nearly Doubles Without Training
机器之心 · wechat · 2026-08-30
Research from Salesforce AI and UIUC proposes Strong-to-Weak Scaffolding: a stronger Builder model automatically designs a Harness—task routing, deterministic solvers, explicit state tracking, format checks—for a weaker Target model, with only 5% of each benchmark exposed as validation data and no parameter changes to the Target.
With GPT-5.4-mini as Target, raw accuracy averages 0.488, while auto-built Harnesses average 0.763 (best 0.912); all 57 scaffold runs beat baseline, even surpassing the raw stronger GPT-5.4 (0.619). The paper frames this as cognitive load reduction: what transfers is not knowledge but the Builder's understanding of task structure compiled into external scaffolding—the share of work done by code correlates strongly with accuracy (r=0.72). Not all reasoning compiles into rules (94% of BigToM vs 36% of MuMA-ToM), weaker Targets gain more, and Builder reasoning effort matters more than iterative refinement—suggesting evaluation should shift from bare model capability to model-plus-environment system capability.
Related event: Strong-to-Weak Scaffolding Nearly Doubles Small Model Accuracy(2 posts)→
More from coding & agent
- Agent named 'Ilya' reportedly scores 1.1pt higher on benchmarks — daniel_mac8 · 2026-10-16
- Gemini CLI fix: substring matching inlined unrequested binary files into model context — priyanshu1976 · 2026-10-16
- Building AI agents on real healthcare claims data: the domain work that makes them work — alex_verem · 2026-10-16
- OpenAI stealth-testing mysterious internal model 'Aeon' in Codex desktop app and Dots — cedric_chee · 2026-10-16
- Vibe-coded iPhone Duo sniping bot built with Codex in 17 minutes, 41 tests passing — xiaohu · 2026-10-16
- Long-term multiplayer agent memory tool tops BEAM memory benchmark — CurieuxExplorer · 2026-10-16