Open-source CLI audits agent traces for wasted tokens — one agent burned 35% of I/O re-verifying itself
Impressive_Laugh_176 · reddit · 2026-10-20
The author audited 73 public benchmark agent trajectories — 74% carried measurable token waste — and released chewme-audit (MIT), a local CLI that reads Langfuse, LangSmith, JSONL and Claude Code traces, prices the waste per provider, and suggests fixes.
Key findings
- One trajectory re-sent the same malformed edit command 283 times; a dedup fix cut waste 77% (13,193 → 3,041 tokens).
- Live solo agent: 35% of I/O was re-verification (same 'find files' script run 7 times).
- Four identical coordinator-free agents: 15–37% waste; one deleted 117 lines of real docs after skimming files with head -3.
- A debug run passed 12/12 for $0.013, but the trace shows the fix came from an accidental git checkout -- — a clean rerun cost $0.0077. Pass/fail alone would have certified a false success.
Cost per million wasted tokens (cache-miss pricing, Oct 2026): DeepSeek $0.15, Gemini $0.75, Grok $1.25, Claude Sonnet $2.00, GPT-5 class $4.00. Limitations honestly flagged: cache and reasoning-token overspend require API logs; all dollar figures labeled estimated.
Related event: Open-Source Audit of 73 Agent Traces Finds 74% Waste Tokens(2 posts)→
More from coding & agent
- ClaudeST: an open-source native Claude client for Atari ST retro computers — Hakan_Ozalp · 2026-10-20
- OpenCode: the open-source Claude Code rival that runs GPT, Gemini or local models — thisdudelikesAI · 2026-10-20
- DevDay demo uses Decisions API to detect tool use inside live voice conversations — gabrielchua · 2026-10-20
- GPT Live + Decisions API powers fully voice-driven slide demo at OpenAI DevDay — gabrielchua · 2026-10-20
- Decisions API integration with Live API showcased as fast GPT backend in demo — gabrielchua · 2026-10-20
- DoorDash runs 130K automated tasks a month: inside 8 AI software factories — Pavan_Belagatti · 2026-10-20