STAPLE paper: sparse-memory upcycling boosts OLMo-2 HumanEval by 4.27 with 2× faster decoding
teortaxesTex · x · 2026-10-20
A CMU/Amazon/Meta paper introduces STAPLE, a recipe that adds token-indexed embedding tables as a residual path inside pretrained FFNs instead of replacing them. On OLMo-2 1B it beats continued dense training by +4.27 HumanEval, +2.80 MBPP, +2.27 GSM8K, outperforms STEM and Engram-style baselines, and decodes 2× faster than an MoE upcycling baseline with under half the GPU memory. Embedding tables live on CPU with only 5% of tokens cached, keeping throughput within 1.5–3% of dense. Code is open-sourced.
Related event: STAPLE: Sparse Memory Upcycling for Pretrained Models(2 posts)→
More from Infra
- Liquid AI Launches RIPL: Ultra-Low-Latency Inference Engine With Sub-50ms End-to-End Responses — JosephJacks_ · 2026-10-23
- OpenJet claims Qwen3.8-27B coding runs 1.9x faster than llama.cpp on one RTX 3090 — JeremyCMorgan · 2026-10-23
- Jensen Huang lays out NVIDIA's full Vera Rubin agentic system: NVL72, MGX, CMX and RTX — rohanpaul_ai · 2026-10-23
- ONNX to Transformers.js: running models directly in your browser — JosephJacks_ · 2026-10-23
- MiniMax M3.1 Flash free for 3 days on 0G, with 1M context and 512K output — MiniMax_AI · 2026-10-23
- "We bet on Cloudflare and it paid off": R2 and D1 at billion scale — dinasaur_404 · 2026-10-23