DeepSeek Harness Eval Bug Triggers Agent Infinite Loop, Tripling Cost
teortaxesTex · x · 2026-08-14
While evaluating DeepSeek Harness (dsh), BohuTANG discovered a critical agent policy bug: when grep returns an exit code 1 for zero matches, dsh treats it as a failure requiring investigation. This causes the agent to continuously create and fix additional verification steps even after tests have passed, trapping it in an infinite loop.
Under identical task, model, and reasoning configurations, this bug leads to massive resource waste:
- Requests: 61 vs normal 32
- Duration: 10m38s vs 4m55s
- Cost: $0.17 vs $0.05
- Input tokens: 2.9M vs 548K
More from coding & agent
- AI Skills Can't Replace People Yet, But Bosses Think They Can — lxfater · 2026-08-14
- Grok Generates Playable Candy Game From a Single Prompt — Daniel_Farinax · 2026-08-14
- OpenMausBot Open-Source Project Hits 800+ Stars, Declares No Crypto Affiliation — aigclink · 2026-08-14
- OpenMausBot: An Open-Source Grok Bot Running Entirely Local — aigclink · 2026-08-14
- Dev Launches Open Multi-Node Network for AI Agents with On-Chain Treasury Support — KidneeBean · 2026-08-14
- Hands-on with Grok Computer Use Agent: Bridging Legacy Software — yunta_tsai · 2026-08-14