EdgeRock-4B cuts KV cache 99.1%: constant 72MiB at 256K context, 6.4x throughput, open-sourced
量子位 · wechat · 2026-10-19
Rockmatrix (Yanshan Tech) with HKUST and CMU released EdgeRock-4B, an open-source model whose KV cache stays constant at 72MiB even at 256K context — a 99.1% reduction versus the source model.
Key results: 512 concurrent 32K+32K long-reasoning requests on a single 80GB Hopper GPU at 12,124 tokens/s output (6.42x Qwen3.5-4B), while scoring 82.9% average pass@1 on AIME 2024–2026 — far above bounded-state baselines where the best 14B model hit 18.8% — and retaining 96% of the source model's math reasoning.
The mechanism, Rock Memory Anchors (RMA), splits history into a recurrent state, a 1024-token local window, and a fixed 1024-entry anchor library curated by a writer/scorer pipeline. Disabling RMA drops long-context QA accuracy from 94.1% to 0.4%, confirming anchors drive remote recall. Converted from Qwen3.5-4B without pretraining, weights and code are released for commercial use.
More from Infra
- OpenJet claims Qwen3.8-27B coding runs 1.9x faster than llama.cpp on one RTX 3090 — JeremyCMorgan · 2026-10-23
- Jensen Huang lays out NVIDIA's full Vera Rubin agentic system: NVL72, MGX, CMX and RTX — rohanpaul_ai · 2026-10-23
- ONNX to Transformers.js: running models directly in your browser — JosephJacks_ · 2026-10-23
- MiniMax M3.1 Flash free for 3 days on 0G, with 1M context and 512K output — MiniMax_AI · 2026-10-23
- "We bet on Cloudflare and it paid off": R2 and D1 at billion scale — dinasaur_404 · 2026-10-23
- Public data is drying up; private training data is turning into a browsable catalog — SucceededMind · 2026-10-23