A PyTorch-like framework for training reusable agent harnesses
Megadragon9 · reddit · 2026-07-21
A Reddit post introduces harness-training, a PyTorch-like framework for training a reusable agent harness rather than a single task model.
The core idea is to train the harness once against a fixed task LLM and environment, then swap in different task LLMs later and evaluate the same frozen harness across new environments. The author says the current implementation supports any OpenAI-compatible API and can train against Terminal-Bench or SWE-Bench, with the framework designed to be extensible to other task environments.
The post also explains the training loop: a baseline-vs-candidate verdict is treated like loss, successful candidate changes are promoted via git commits, rejected ones remain as refs, and the project aims to improve general capability transfer across models and tasks. The accompanying blog post reports that the framework improved results on Terminal Bench 2.0 and even transferred from SWE-Bench-style training to terminal tasks. A key lesson from the work was that determinism was missing in the original design.
Related event: Research Suggests RLM Generalization is Driven by External Harness(10 posts)→
More from coding & agent
- BELL: Open-Source Stack Combines Browserbase, Eve, iMessage and Stripe Link for Personal Agents — jeff_weinstein · 2026-10-27
- Legora runs 2,000 parallel agent evals on Modal Sandboxes for legal AI — blelbach · 2026-10-27
- Two engineers rebuilt a FedEx supplier's billing core in 3.5 months vs an estimated 7-8 — alex_verem · 2026-10-27
- GPT-6.1 Terra gains ~20 points on Terminal-Bench 4.0, rivalling GPT-6.1 Sol — ArtificialAnlys · 2026-10-27
- GPT-6.1 Terra scores 48 on AA Coding Agent Index at one-third the cost of GPT-5.6 Terra — ArtificialAnlys · 2026-10-27
- Shift-1 router cuts coding agent spend up to 55% while keeping SWE-bench accuracy — ypatil125 · 2026-10-27