DeepSeek's V4.1 paper reveals KV cache reuse across checkpoints — 'a very strange model factory'
teortaxesTex · x · 2026-09-10
Digging into DeepSeek's V4.1 paper, Astra Pro surfaced several intriguing details — most notably that DeepSeek reuses KV caches between checkpoints. Commentator teortaxesTex calls the practice highly unusual, saying DeepSeek has 'a very strange model factory going,' hinting at an unconventional training pipeline.
Related event: DeepSeek V4.1 Flash Confirmed to Center on YOCO Architecture(7 posts)→
More from Research
- Rewriting scaling law study: optimal synthetic data share rises from 10% to 32% with compute — heghbalz · 2026-10-26
- FLASH: GPU-native deformable sim learns cloth folding in minutes on one RTX 5090 — ChongZzZhang · 2026-10-26
- LEMON: zero-shot LLM misbehavior monitor flags hallucinations from attention maps — yoavartzi · 2026-10-26
- Behind the EMNLP keynote: LMs as simulated learners to generate child language acquisition hypotheses — najoungkim · 2026-10-26
- The paper behind the EMNLP 2026 keynote on bigram 'simulated learners' — najoungkim · 2026-10-26
- Laying out DeepLIFT as a standalone computation graph: from sequence to attributions directly — jmschreiber91 · 2026-10-26