Same model, half the tokens: chaff, an open-source TypeScript agent harness
uriwa · reddit · 2026-11-22
The author open-sourced chaff, the MIT-licensed TypeScript agent harness powering prompt2bot in production (npm: @uriv/chaff, runs on Node 20+, Bun, Deno, Cloudflare Workers). Thesis: people benchmark frontier models constantly while treating the harness as neutral plumbing — hold the model constant and the harness gap is huge.
Benchmarks (ClawBench t2-config-loader, same model vs pi-coding-agent): both passed, but chaff used 800 vs 4,239 turn-1 startup tokens (-81.1%), 944 vs 8,711 output tokens (-89.2%), 120.9k vs 248.8k total (-51.4%), 16 vs 36 LLM calls. On tau-bench Retail (5/5) vs the reference LLMAgent: -66.6% output tokens, 55.2% vs 0% prompt cache hit rate.
Savings come from structural choices, not "be concise" prompting: a typed multi-party event log separating tool execution from outward messages; dynamic skill loading/auto-unlearning keeping unused schemas out of declarations; three-tier compaction that protects cache prefixes (fold results in memory, spill large outputs to a paged SCRATCH: scratchpad, roll up settled sessions); per-turn LOW/MEDIUM/HIGH thinking routing; zero-cost deterministic happy-path guards; and preserving thinking metadata pairing so thinking models don't loop on tool calls.
Related event: Open-Source Chaff Framework Cuts Group-Chat Agent Token Output by 89%(2 posts)→
More from coding & agent
- Dev builds auto-generated collaborative canvas to iterate on creative work with AI agents — max__drake · 2026-11-22
- GPT-6.2 breaks Responses API prompt caching: hit rate falls from 97.4% to 46.4% — lxfater · 2026-11-22
- Rustingstrata + DSH balloons prompts to 195k tokens while llama.cpp works fine — Mordimer86 · 2026-11-22
- Open-source Laya: single-pass System 1 decision engine that keeps learning from LLMs — solyarisoftware · 2026-11-22
- CTO merges branch 243 commits ahead of main, flexing second GitHub YOLO badge — mayanicks0x · 2026-11-22
- Dev uses Codex to relight Elden Ring into a next-gen look — runs on a 2016 GTX 1070 — banteg · 2026-11-22