Strong models build harnesses for weak ones, nearly doubling benchmark accuracy without training
青稞AI · wechat · 2026-09-03
A writeup of the paper "AI4AI at Test-Time: Strong-to-Weak Capability Transfer via Harnesses": instead of distillation or fine-tuning, a strong model acts as a builder that automatically constructs inference-time harnesses—task routing, structured extraction, deterministic code, verification, output control—for a weaker target model with zero parameter updates.
- Just changing the external harness nearly doubles GPT's accuracy across benchmarks.
- Gains come from compiling part of the reasoning into code, rules, and structured procedures rather than making the weak model think longer; the more reasoning offloaded and the stronger the builder, the better the results.
- Frames an "AI4AI" thesis: capability can live outside weights as tools, code, routing, and verification environments.
The post also promotes a livestream talk by UIUC PhD student Qian Cheng on Sept 5.
More from coding & agent
- What thousands of hours of agent runs reveal about LLM-as-judge: good at progress, bad at safety — hrishioa · 2026-09-21
- Do you watch every agent tool call? One dev says he watches the full diff stream — BLUECOW009 · 2026-09-21
- Agent designs its own hardware: Astra produces USB-powered 4-light PCB files — paraschopra · 2026-09-21
- Open-source JEV router cuts LLM costs from $0.24 to $0.0001 per query with 300ms routing — 1337NET · 2026-09-21
- Building a personal memory: screenshot every 5s, OCR it, ask and get links in seconds — altryne · 2026-09-21
- Upgrading an n8n automation to sync per-repo GitHub commit stats into Obsidian — ColleenMBrady · 2026-09-21