STAPLE: Sparse Memory Upcycling for Pretrained Models

Researchers from CMU, Amazon and Meta introduced STAPLE, which upcycles pretrained checkpoints by adding sparse memory via embedding modules instead of full retraining, boosting OLMo-2's HumanEval score by 4.27 points.

2026-10-20 ~ 2026-10-20 · 2 related posts

• STAPLE paper: sparse-memory upcycling boosts OLMo-2 HumanEval by 4.27 with 2× faster decoding — teortaxesTex · 2026-10-20
• STAPLE paper shows you can cheaply sparse-upcycle pretrained checkpoints with embeddings — teortaxesTex · 2026-10-20