RoastMyHarness: open-source tool benchmarks your custom Pi agent setup against bare Pi on DeepSWE
AnotherObsceneBean · reddit · 2026-09-13
A Redditor open-sourced RoastMyHarness (github.com/sij0sh/roastmyharness), a small engine that runs DeepSWE benchmark tasks comparing bare Pi as a control against your custom harness, extension, skill, or AGENTS.md, showing what improved, what broke, and what it cost in tokens. Includes a wizard Pi extension for easy setup. The author warns beating base Pi on task quality/token efficiency is harder than you'd think. WIP, tested on Linux only.
More from coding & agent
- 16 Calls in One Hour: A Metacognitive Loop That Knows When to Stop Self-Correcting — Dzikula · 2026-09-13
- Stanford launches CS329Z, a new fall course on engineering AI agents — jyangballin · 2026-09-13
- Draw Things launches Local Code beta: local coding agents on Mac at ~980 tok/s prefill — liuliu · 2026-09-13
- Agents get stronger weekly yet still solve the wrong problem: AI takeover ETA? — BarriosA2I · 2026-09-13
- AI assistants fix the crashing line, not the system state that keeps producing it — ClickOk5811 · 2026-09-13
- Swarms revamps chat UI with agent controls and 2,000+ models via its API — KyeGomezB · 2026-09-13