Open Source Tool Slashes 87% of LLM API Input Tokens
naruto_uzumaki00 · reddit · 2026-07-05
A developer open-sourced a vendor-neutral token optimization layer, serving Claude Code/Codex via a proxy + MCP plugin. On real billed tokens for GPT-5.5 and Opus 4.8, inputs dropped from 16875/26573 to 2232/3343 respectively (an 86.8%/87.4% reduction), while task quality held steady at 3/3.
The stack relies on four core mechanisms: prefix caching, tail compression with numeric guards, AST/text retrieval, and cascading verifiable steps to a local model. The author transparently lists limitations: static embeddings didn't beat keyword retrieval, the 3B compressor occasionally dropped critical numbers, and the evaluation sample size was small and biased favorably. Released under Apache-2.0 with reproducible benchmarks.
Related event: Open MCP Server Slashes LLM Input Tokens by 87%(2 posts)→
More from coding & agent
- Victor Taelin updates his agent memory system with tree expansion and long-term recall — AccBalanced · 2026-07-27
- Microsoft’s ReOPD reuses teacher prefixes to make agent distillation 4× faster — dair_ai · 2026-07-27
- One developer merges Claude Code and Codex histories into a 10-million-message archive — doodlestein · 2026-07-27
- Google roundup lists 15 free AI tools across marketing, coding, docs and music — aigclink · 2026-07-27
- A 9B Ollama agent can run a fully local DJ radio with tools, memory, and TTS — pinku1 · 2026-07-27
- Bugbot rejects an MCP permission flag because it would break path-scoped isolation — zeeg · 2026-07-27