EdgeRock-4B locks推理 memory at 97.76 MiB per request while scoring 82.9% on AIME
xiaohu · x · 2026-10-23
EdgeRock-4B introduces a fixed-memory reasoning architecture: no matter how long the chain of thought or how large the context grows, each request's attention and state memory stays constant at 97.76 MiB (KV cache just 72 MiB).
Benchmarks
- 82.9% on AIME math competition, nearly matching top small models with unbounded KV growth
- At 256K context, attention cache drops from 8 GB to 72 MB (99.1% reduction)
- A single 80GB GPU can run 512 concurrent long-reasoning tasks, at 4.9–6.4x the throughput of standard architectures
How it works (three parts)
- Local window: precise view of the most recent 1,024 tokens for fluent generation
- Recurrent impression table: fixed-size state tracking the gist of the context
- Rock-solid Memory Anchors (RMA): a fixed 1,024-slot 'note box'; a dedicated module scores each candidate memory and periodically evicts low-value entries, keeping only high-value clues retrievable verbatim
The chart shows EdgeRock-4B beats Qwen3.5-4B by 5–17 points on MBPP, HumanEval and SuperGPQA, but trails by 10+ points on HellaSwag, LiveCodeBench and GPQA-Diamond — the gains aren't free.
Related event: EdgeRock Open-Sources Constant-Memory Multimodal Inference for Edge Devices(4 posts)→
More from Research
- DeepMind's IPO Paper Gets a 1000x Citation 'Hack' — Thanks to Another Paper Named IPO — _lewtun · 2026-10-23
- Why LLMs dissolve the long-standing barriers to formally verified software — teortaxesTex · 2026-10-23
- Japan's Moonshot IoB Brain Project Publishes 5-Year Research Posters Online — kaixhin · 2026-10-23
- Agent evals should test metric definition drift, argues dev in new post — jylusdev · 2026-10-23
- ruff author asks: how hard is scaling AI chip design from RISC-V to x86? — charliermarsh · 2026-10-23
- Transformer jointly models electrons and magnetism with 0.01% energy error — bravo_abad · 2026-10-23