DeepSeek V4 training details: RL beyond collapse, WSD schedule, 1M context over 10T tokens
stochasticchasm · x · 2026-09-11
Community analysis suggests DeepSeek V4 is the second datapoint after MAI-thinking-1 of models continuing RL training beyond collapse — and, absurdly, merging checkpoints across different scaffolds/harnesses.
Follow-up notes: no instabilities observed, suggesting whatever issue arose during v4 training was solved — a good sign for the architecture; the team kept the WSD schedule and extended context length over 10T tokens, with training at 1M tokens for that long being quite wild. Unofficial analysis, unconfirmed by DeepSeek.
Related event: DeepSeek V4 training details: 10T+ tokens, 1M context, stable RL(3 posts)→
More from Models
- ToneBench eval: GPT-6.1 Terra ranks #23 at writing — cheaper, not better, than GPT-5.6 Terra — OnlyProggingForFun · 2026-10-28
- Cursor user complains it silently switches to Grok, which fails basic tasks — kylegawley · 2026-10-28
- Musk says Grok Bot keeps improving, calls Vault feature a game changer — beffjezos · 2026-10-28
- Claude refuses to gather legitimately public information, sparking guardrail backlash — mkheck · 2026-10-28
- DesignArena launches Code to Video Arena: models direct 15-45s films purely in code — yshan2u · 2026-10-28
- ChatGPT's 'yes, but with an important clarification' often precedes pure nonsense — Crixusgannicus · 2026-10-28