Benchmarked token-saving tools RTK vs Headroom: barely cheaper, both together cost most
Delicious-Flan88 · reddit · 2026-09-09
A controlled test of the two most-starred token-saving repos, RTK and Headroom, on GPT-6 Astra: 5 Rust coding tasks, 4 setups (no tool / RTK / Headroom / both), 3 runs each — 60 sessions, all correct, priced with caching.
- No tool: $0.154; RTK: $0.139; Headroom: $0.158; both: $0.175 (most expensive)
- RTK won consistently only on a multi-file grep task (87 lines grouped into 29, needed line preserved)
- Headroom won its target JSON task 3/3 runs ($0.0054 vs $0.0066)
- Elsewhere costs varied more with the model's own command choices than tools: 9 of 20 cells within 10%, one spread by 13x
- All commands and raw-vs-RTK outputs published for verification
Related event: Tests Show Token-Saving Tools Like RTK Fail to Cut AI Bills(2 posts)→
More from coding & agent
- HABIT-Bench: Best agent memory system scores just 33.61% at inferring user habits — SJTU · 2026-10-28
- Regehr's 'Piano Test': would you trust formally verified code with your life? — moyix · 2026-10-28
- Vibe-coded 'AI 浮生记' game satirizes AI startup life with compute black-market trading — vista8 · 2026-10-28
- The legendary fake Go generics: Unicode Canadian Syllabics as angle brackets — rickasaurus · 2026-10-28
- Microsoft Agensh board: a coding agent's "tests passed" is just a message — shoumikgoswami · 2026-10-28
- Ollama and FriendliAI to Host Seoul Meetup on Building Production Agents with Open Models — ollama · 2026-10-28