Integer multiplication repo becomes an agent harness benchmark — and collaboration loses
iskander · x · 2026-10-14
RohanArun and colleagues found the integer multiplication optimization repo doubles as a new benchmark for agent harnesses: rewind to an older, easier branch, then pit harnesses/prompts against each other to see who pushes the frontier fastest. Forcing genuinely novel solutions makes it ungameable.
The counterintuitive finding: sharing context and collaborating is bad. A 1-vs-2-vs-3-vs-4 agent matrix shows a single agent submitted in 5m16s with zero handoffs, while multi-agent setups were slower and far more token-hungry (one 2-agent run hit a 60-minute cap). Parallel independent agents won — possibly explaining why swarms were favored for the Navier–Stokes solution.
Contamination controls matter: inspect traces for web search, disable it, and ensure checkpoints don't include published solutions. Results are being scaled up. Community auto-research progress since 10/11 is +28.9%, with 85 first-time PR contributors in the last 4 hours.
More from coding & agent
- kimo v3: Open-Source GUI-less Video Editor Where an Agent Cuts a Full 2-Minute Video via MCP — LieTop7965 · 2026-10-14
- Clayzo launches: a lightweight WebGL animation engine built for coding agents — CurieuxExplorer · 2026-10-14
- 50 practical tips for working with AI: context engineering, voice, and full toolchain control — koltregaskes · 2026-10-14
- Agor Open Beta: Run Multiple Agent Harnesses on Shared Spatial Boards — rohanpaul_ai · 2026-10-14
- A 12-month roadmap to becoming an AI engineer in 2026 without a CS degree — mdancho84 · 2026-10-14
- TV Time's shutdown cost him 15 years of data, so he rebuilt a 10,000-item media tracker in Obsidian — dSebastien · 2026-10-14