Bare IPython Harness Matches Production Agents on Qwen3.5-9B; Output Normalization Kills Run-to-Run Variance
HauntingMoment · reddit · 2026-10-19
Testing Qwen3.5-9B on Terminal-Bench Lite, a single-tool IPython harness matched heavier production harnesses (QwenCode, OpenHands, opencode, Pi). Much of small-model agent "randomness" comes from environment noise: normalizing tool outputs like download speeds with 40 lines of regex plus quieting the environment lifted pass rate from 70% to 95%, cut tokens per attempt from 130k to 50k, and eliminated divergence at temperature 0. All harnesses spent 25-30% of calls reading files, driven by system prompts rather than the model. Takeaways for local agent builders: less scaffolding, control sampling params, scrub tool-output noise. Harness is open source.
More from coding & agent
- Anthropic Built Scion, Its Own Source Control System, as Merged Changes Grew 12x to 30,000/Day — Pavan_Belagatti · 2026-10-19
- No editing software: one creator made a full AI video just by chatting with Claude — aziz4ai · 2026-10-19
- 20+ exposed panels reveal open-source AI hacking agent Cairn in real cyberattacks — Hesamation · 2026-10-19
- The Vibe Coding Curse: Understanding AI Code Defeats the Speed, Not Trusting It Blocks Shipping — MoonL88537 · 2026-10-19
- Six months of agent-memory provenance fields nobody could see: a Reddit stranger found the gap — Neither-Witness-6010 · 2026-10-19
- The AI coding curse: skipping code understanding makes you helplessly dependent on the model — MoonL88537 · 2026-10-19