Distil adds statistically gated context compression for LLM agents and ties full context on SWE-bench
chandu1221 · reddit · 2026-07-26
Distil adds statistically gated context compression for agents
The project introduces Distil, a context compressor for LLM agents that does not rely on blind trust. Instead of assuming shorter context is safe, it runs each compression setting through a paired non-inferiority test (TOST/McNemar) on real agent trajectories and rejects any setting that changes tool calls or outputs.
Key claims:
- Stdlib-only and zero runtime dependencies; works on stock Python 3.9+
- A reversible tier can recover the original bytes on demand
- Can run as a proxy for Anthropic, OpenAI, and Gemini-style APIs
- On 500 SWE-bench Verified tasks, the compressed run scored 42.0% vs 39.2% for full context, which the author describes as a certified tie because the confidence interval includes zero
The author says an earlier version failed in practice: aggressive summarization looked good per turn, but end-to-end agent success collapsed. They also learned that rewriting the top of the prompt can invalidate prefix caching, so Distil keeps the prefix byte-stable and only compresses the tail.
A no-key demo is available via:
bash
uvx --from distil-llm distil bench
Repo: dshakes/distil.
More from coding & agent
- Vjeux says coding loops still fail outside training data unless a human babysits them — Vjeux · 2026-07-26
- Hugging Face’s breach analysis hit closed-model guardrails, then worked with an open model — mmitchell_ai · 2026-07-26
- LocalMind brings local AI memory to FastMCP, Ollama, and NixOS — xmrah · 2026-07-26
- Agent harness discussion on Max Agency leaves builders wanting to redesign everything — hwchase17 · 2026-07-26
- Building an MCP Server with 33 Tools: Lessons from Designing for Claude — SimpleSpacer97 · 2026-07-26
- Multi-agent workflows need checkpoints, shared state, and circuit breakers — blaizedsouza · 2026-07-26