Open-source chaff framework rethinks agents for group chats: 89% fewer output tokens, 55% cache hits
uriwa · reddit · 2026-11-22
Running agents in real messaging channels and live voice calls led the author to open-source chaff (@uriv/chaff, MIT), built on the thesis that most harnesses assume a 1-on-1 terminal loop and fall apart in group chats.
Key production-tested design choices:
- First-class silence: typed events with speaker, timestamp, timezone; the agent can emit donothing when humans are talking to each other
- Time-aware threads: sessions split on 30-minute gaps; settled sessions roll into structured summaries without breaking tool-call/result pairs, preserving prompt cache prefixes
- Skills that turn off: a decision router loads only relevant tool schemas per turn instead of bloating every turn
- One loop for IM and full-duplex voice with barge-in handling
- No tuning knobs: thinking-level routing, scratchpad spills, and deterministic guards for repetition loops run automatically
Numbers: vs Pi on ClawBench at constant model, cutting startup/narration bloat cut output tokens by 89.2% (944 vs 8,711) and total tokens by 51.4%; on tau-bench Retail, deterministic history prefixes took prompt cache hit rate from 0% to 55.2%.
Related event: Open-Source Chaff Framework Cuts Group-Chat Agent Token Output by 89%(2 posts)→
More from coding & agent
- Dev builds auto-generated collaborative canvas to iterate on creative work with AI agents — max__drake · 2026-11-22
- GPT-6.2 breaks Responses API prompt caching: hit rate falls from 97.4% to 46.4% — lxfater · 2026-11-22
- Rustingstrata + DSH balloons prompts to 195k tokens while llama.cpp works fine — Mordimer86 · 2026-11-22
- Open-source Laya: single-pass System 1 decision engine that keeps learning from LLMs — solyarisoftware · 2026-11-22
- CTO merges branch 243 commits ahead of main, flexing second GitHub YOLO badge — mayanicks0x · 2026-11-22
- Dev uses Codex to relight Elden Ring into a next-gen look — runs on a 2016 GTX 1070 — banteg · 2026-11-22