One harness trained on a small model lifts Terminal-Bench scores across four models
Megadragon9 · reddit · 2026-07-24
Training the harness on one small model lifts Terminal-Bench scores across four others
A developer reports that training an agent harness—not the base LLM itself—improved Terminal-Bench 2.0 scores for four different models, including DeepSeek v3.2, GPT-OSS 20B/120B, and MiniMax M2.5, and even beat the official Terminus 2 harness.
The setup freezes the model and optimizes the surrounding system:
- prompts
- tools
- context management
- repair loops
The harness lives in a single Python file, and the training loop proposes changes, then promotes them only if they meet a quantitative criterion.
The author trained the harness once using a small Qwen 3.6 35B A3B NVFP4 model hosted on rented 5090 GPUs. They then used that trained harness across different task environments, arguing that the gains transfer beyond the model used for training.
The accompanying framework is presented as a more general PyTorch-like training loop for agent harnesses, with OpenAI-compatible backends and support for Terminal-Bench or SWE-Bench, plus extension points for other environments. The post also says the biggest missing ingredient in the first version was determinism.
More from coding & agent
- Warp's six non-engineering teams all run on Linear and Claude Code — mon__lim · 2026-09-11
- Is inference latency becoming the biggest bottleneck for production AI agents? — Euphoric_Sea632 · 2026-09-11
- Anthropic researcher: 99% of engineers now run swarms of 300+ self-improving agents — AlishaOutridge · 2026-09-11
- Gergely Orosz: Shipping 10x PRs With AI Agents, Sites Fill With Small Regressions — ducha_aiki · 2026-09-11
- Same Echo Maze prompt, three frontier models: all passed visually but shipped the same hidden bug — eyishazyer · 2026-09-11
- Astra storyboards plus Minimax H3 per-shot generation boost video success rates — Hailuo_AI · 2026-09-11