DeepSeek keeps moving the quadratic component up, a 'bitter lesson' science play
teortaxesTex · x · 2026-09-16
teortaxesTex argues DeepSeek's architecture research deliberately moves the quadratic component into ever rarer operations, accepting the bitter lesson in search of the minimal sufficient primitive of global computation. He says they ignore linear attention not out of ignorance — they read Kimi's papers and adopted Muon — but because it conflicts with this infra-driven scientific project.
Related event: DeepSeek V4.1 Flash resets architecture, shrinking KV cache to a quarter(5 posts)→
More from Models
- AWS's open-source Strands Harness claims 45% cheaper than Claude Code, reigniting harness debate — blaizedsouza · 2026-11-02
- Yacine calls Terra High 'kind of an insane model' — yacineMTB · 2026-11-02
- Running DeepSeek V4.1 on a single H200 at $5.30/hr — daluoseo · 2026-11-02
- Leaked Fable 5.5 checkpoint video called 'pure code' — 'will change the game' — kimmonismus · 2026-11-02
- Claude Opus 5.5 quality complaints spark debate on why continual learning is so hard — JeremyNguyenPhD · 2026-11-02
- Why is AI sexist? Reddit debate on models overlooking Iceland's women's strike — Mathemodel · 2026-11-02