Same Model, Different Harness: Coding-agent results vary by context policy
rohanpaul_ai · x · 2026-08-31
A study compares Yuj coding agent configurations: a control keeping full history vs. a treatment compressing old tool outputs and detecting stalls. On SWE-bench Verified with a 20k token window, Qwen2.5-Coder's mean F2PF rose from 28% to 49%, and complete solutions increased from 43 to 72. Benefits vanished at 262k tokens. The research suggests evaluating the model and harness as a unified solver.
Related event: Harness Optimization Doubles Qwen's SWE-bench Performance(2 posts)→
More from coding & agent
- Vibe-coded dashboard leaked franchise financials before anyone logged in — shauntrennery · 2026-10-16
- Fintech HR built a sourcing agent on 60,000 past applicants — 30 qualified candidates in a month — G0NZ0BOY · 2026-10-16
- Orchestrating a 100-image portrait of humanity for aliens with ChatGPT, Argon and p5.brush — dx8xb · 2026-10-16
- 3x DGX Spark cluster runs GLM-5.3-Flash locally behind a proxy, 1.75M tokens/day at 119W — Teknium · 2026-10-16
- Builder's one rule for trusted desktop agents: read freely, but sending, deleting, buying always ask first — No_Fan5310 · 2026-10-16
- GPT-built unifi-simulator makes MikroTik switches appear inside UniFi Network — Pytlicek · 2026-10-16