Download-speed digits break small-model agents: a 40-line regex normalizer fixes it
yb2698 · x · 2026-10-14
The author shows that tool outputs carry stochastic noise — e.g. pip progress bars show different download speeds each run — and small models aren't robust to these artifact changes, degrading agent (harness+model) performance.
The fix: a 'normaliser' applying regex filtering to tool outputs, plus a quiet env to strip artifacts — about 40 lines of Python, now open-sourced.
Iterating from pandas-etl trajectory analysis across harness versions, benchmarks show Qwen3.5-9B going from ipython-v0 to v3: reward 0.70→0.95, input tokens 530k→52k, matching the best external harness (openhands) with 34% fewer input tokens. Lesson: when building harnesses for small LMs, watch out for these environment tricks; large models are likely unaffected.
More from coding & agent
- Google's ADK brings one agent framework to Python, Go, TypeScript, Java and Kotlin — rseroter · 2026-10-15
- AI agent flags a bizarre '71 >= 71' error while a server pool balloons by 1.4TB — haydendevs · 2026-10-15
- Invideo launches an MCP for editing videos directly via Claude or ChatGPT — azed_ai · 2026-10-15
- Connecting Grok to ROS via OpenClaw and AgenticROS for robotics missions — chrismatthieu · 2026-10-15
- Automating bug report fixes: a Bot triages DMs into a Cursor project — msfeldstein · 2026-10-15
- Devin now auto-fixes Pylon support tickets; new customers get $15,000 in credits — DevinAI · 2026-10-15