Open Source Tool Slashes 87% of LLM API Input Tokens
naruto_uzumaki00 · reddit · 2026-07-05
A developer open-sourced a vendor-neutral token optimization layer, serving Claude Code/Codex via a proxy + MCP plugin. On real billed tokens for GPT-5.5 and Opus 4.8, inputs dropped from 16875/26573 to 2232/3343 respectively (an 86.8%/87.4% reduction), while task quality held steady at 3/3.
The stack relies on four core mechanisms: prefix caching, tail compression with numeric guards, AST/text retrieval, and cascading verifiable steps to a local model. The author transparently lists limitations: static embeddings didn't beat keyword retrieval, the 3B compressor occasionally dropped critical numbers, and the evaluation sample size was small and biased favorably. Released under Apache-2.0 with reproducible benchmarks.
Related event: Open MCP Server Slashes LLM Input Tokens by 87%(2 posts)→
More from coding & agent
- alphaXiv open-sources OpenResearch to run parallel research agents with any model — alphaXiv · 2026-09-11
- MathModelAgent gains traction: auto-solves math modeling and writes a submission-ready paper — jihe520 · 2026-09-11
- DeskcommCRM: open-source AI sales CRM with native agents and WhatsApp hits 1k stars — melgarafael · 2026-09-11
- hyperresearch: agent-driven knowledge base that turns web research into a searchable wiki — jordan-gibbs · 2026-09-11
- Forter's 13 lessons from its agent sprint: skip custom RAG, lean on mature enterprise search — bibryam · 2026-09-11
- Two real 'company brains' opened up live: Gorgias' in-house Cortex vs Slite — femke_plantinga · 2026-09-11