EdgeRock-4B: open-source 4B model keeps KV cache flat at 72MiB over 256K tokens
eyishazyer · x · 2026-11-04
EdgeRock-4B, an open-source 4B model (free for commercial use), tackles the growing KV cache problem for long-running agents. While a typical 4B model like Qwen3.5-4B needs 8 GiB of KV cache at 256K tokens, EdgeRock-4B holds history under 100 MiB at any length.
It uses three fixed-size memory layers:
- Recent: local attention over the last 1,024 tokens in full detail
- State: a recurrent state with fixed-size matrices summarizing the context
- Anchors: Rock Memory Anchors, key details the model chooses to keep, re-picked every 256 tokens (up to 1,024 records per KV group)
The Q4KM quantized model is just 2.94 GB and scores 94.1% query accuracy on a long-context memory test (0.4% with RMA off). Technical report: arXiv 2610.1018.
Related event: EdgeRock-4B Open-Sources Constant-Memory Inference for LLMs(3 posts)→
More from coding & agent
- Useful Work: remote MCP server scans site errors, Claude fixes your code — AlmostSneakers · 2026-11-04
- Neonbin launches: share agent workflows as bins, reuse them via MCP — Ok_Giraffe_7279 · 2026-11-04
- With: an MIT-licensed systems language repo with 4,200+ commits and agentic dev workflow built in — QuixiAI · 2026-11-04
- mcptunnel: open-source tool wraps stdio MCP servers into HTTP endpoints — flrngel · 2026-11-04
- Three Claude Code Core Members Leave Anthropic Within 10 Days Ahead of Potential IPO — haider1 · 2026-11-04
- GitHub team shares ~30 min of Copilot demos, from editor to Slack workflows — lee_stott · 2026-11-04