I audited 73 agent trajectories — 74% had measurable token waste (open-source CLI)
Impressive_Laugh_176 · reddit · 2026-10-20
The author open-sourced chewme-audit (MIT), a local CLI that parses agent traces from Langfuse, LangSmith, JSONL, and Claude Code transcripts, prices wasted tokens per provider, and suggests fixes. Auditing 73 public benchmark trajectories found 74% with measurable waste: one sent the same malformed edit 283 times (dedup fix cut waste 77%); a solo agent spent 35% of I/O re-verifying; four uncoordinated agents wasted 15–37%. A $0.013 debugging run passed 12/12 only because of an accidental git checkout -- — pass/fail metrics can't catch that. Caveats: cache behavior and reasoning overspend need API logs.
Related event: Open-Source Audit of 73 Agent Traces Finds 74% Waste Tokens(2 posts)→
More from coding & agent
- How big should your agent's memory file be? One dev caps it at 200 lines with weekly merges — bayvo · 2026-10-20
- Mistral Vibe CLI 3.0 ships with Rust-rewritten TUI: first frame in 72ms, 60fps scrolling — onetwoval · 2026-10-20
- Your agent isn't dumb, your data is lying: audit with 20-30 real questions before scaling — SUPERSAIIYANNN · 2026-10-20
- Building a photo-based bill-splitter: when reasoning models are worth the cost — HowDevelop · 2026-10-20
- Mila's Recursive Self-Aggregation pushes Gemini 3 Flash near top of ARC-AGI-2 — morgymcg · 2026-10-20
- Sparrow turns MRI reports into structured data on infrastructure you control — andrejusb · 2026-10-20