Headroom: Netflix engineer's open-source context compression layer cuts LLM tokens 60–95%

大模型之路 · wechat · 2026-10-17

Sparked by a $287 API bill, Netflix engineer Tejas Chopra open-sourced Headroom (40k GitHub stars), a middleware layer that deterministically compresses context before it reaches the LLM; users have reportedly saved $700k and 200B+ tokens.
Instead of LLM summarization, it uses structured compressors: ContentRouter routes to SmartCrusher (JSON), CodeCompressor (AST-based), or Kompress-base (general text). In official tests, 10,144 tokens compressed to 1,260 with the FATAL error still found accurately — ideal for ReAct loops where tool outputs balloon each step.
CCR (Compress-Cache-Retrieve) makes compression reversible: originals cached locally in Redis/SQLite, retrievable on demand via a tool, doubling as an audit log.
CacheAligner hashes stable prefixes to reuse provider KV caches — even with 90% of tokens compressed, bills can drop to 1/20 (nginx log example: 65,694 tokens/$4.20 → 5,118/$0.40).
Four integration modes: OpenAI-compatible proxy, library, headroom wrap claude|codex|cursor, and MCP server. Fully local, Apache 2.0, Python+Rust.
Caveats: very short, high-signal context shouldn't be compressed; the sweet spot is long, repetitive, structured content. Compression is orthogonal to context window size — window determines capacity, compression determines signal density.

More from coding & agent