Strong Models Design Harnesses for Weak Ones: Accuracy Nearly Doubles Without Training
机器之心 · wechat · 2026-08-30
Research from Salesforce AI and UIUC proposes Strong-to-Weak Scaffolding: a stronger Builder model automatically designs a Harness—task routing, deterministic solvers, explicit state tracking, format checks—for a weaker Target model, with only 5% of each benchmark exposed as validation data and no parameter changes to the Target.
With GPT-5.4-mini as Target, raw accuracy averages 0.488, while auto-built Harnesses average 0.763 (best 0.912); all 57 scaffold runs beat baseline, even surpassing the raw stronger GPT-5.4 (0.619). The paper frames this as cognitive load reduction: what transfers is not knowledge but the Builder's understanding of task structure compiled into external scaffolding—the share of work done by code correlates strongly with accuracy (r=0.72). Not all reasoning compiles into rules (94% of BigToM vs 36% of MuMA-ToM), weaker Targets gain more, and Builder reasoning effort matters more than iterative refinement—suggesting evaluation should shift from bare model capability to model-plus-environment system capability.
More from coding & agent
- Rapid Evolution of Codex and Claude Code: Live Demos Far Outshine Text Descriptions — tinyfool · 2026-08-30
- Lawyer Builds Fleet of AI 'Executives' With Cursor, Cuts 3-Hour Content Job to 10 Minutes — Specialist_Call_1257 · 2026-08-30
- Robotics Experiment: Claude Coding Failed Completely, Infrastructure Bugs Hinder Progress — verdakorz · 2026-08-30
- Perplexity's Portable Computer: A Local-First Agent on Qwen3.8 27B, Near-Zero Cost — 机器之心 · 2026-08-30
- Dev Uses AI to One-Shot an Android Port of His 11-Year-Old Hand-Coded Wedding Canvas Art — steren · 2026-08-30
- Fully Open Source Stack: Qwen and Hermes Create a Self-Modifying PC Experience — ramagetime · 2026-08-30