DeepSeek-V4 answers flip on a 4-token cycle, traced to chunked KV-cache compression
Francis_YAO_ · x · 2026-10-20
Researchers show DeepSeek-V4 has periodic weak spots: merely prepending decorative docstring tokens flips its answer every two tokens, and NIAH accuracy swings by up to 40 points depending on needle position, on a fixed 4-token cycle.
The preprint attributes this to chunked KV-cache compression: to save memory on long contexts, the model merges windows of W tokens into single cache entries every S tokens, so a token's phase (t mod S) creates periodic sensitivity.
They open-sourced the Periodic-Weak-Spots repo and built a Qwen3-derived chunked KV-compression model to reproduce the phenomenon.
More from Models
- Hazy Research's open-source NCM generates text 31x faster via coarse-to-fine concept modeling — simran_s_arora · 2026-10-20
- Blogger claims DeepSeek's unreleased model nearly matches Opus at design, shares one-prompt demo — sven_ai · 2026-10-20
- Fingerprint test flags DeepSeek ds-4.1-flash as likely GPT-6-astra — sven_ai · 2026-10-20
- Intelligence Saturation: 10-Point Model Edge Now Buys Just 20% More Developer Traffic — khademinori · 2026-10-20
- MiniMax M3.1 Flash Hits 73.3 on BuildingBench, Reshaping the Cost-Quality Frontier — Lianhuiq · 2026-10-20
- User: Codex burns half of the day's usage limits within an hour after banked reset — ssh4net · 2026-10-20