MTP dropped in new model's tech report: fp8 lookup tables and prime table sizes noted
stochasticchasm · x · 2026-09-11
Reading a new model's tech report, the author notes MTP was removed—likely because the uplift wasn't worth it. The engram section reveals unusual details: table sizes chosen as distinct primes, 4-grams not seen in comparable models, and fp8 lookup tables suggesting an inference-side design.
Related event: New Model's KV Cache QAT and Dropped MTP Spark Debate(4 posts)→
More from Models
- Users notice o5 constantly self-measures while f5/5.1 barely does — repligate · 2026-09-11
- Chinese labs' distillation scale revealed: Alibaba 151M+ exchanges, Moonshot 23M+, DeepSeek 12.1M+ — scaling01 · 2026-09-11
- Astra usage limits worse than Fable: burn a week's quota in a single day — cocktailpeanut · 2026-09-11
- DeepSeek V4.1 Flash Architecture: 552B MoE with Asymmetric 8B Read / 16B Decode Compute — demian_ai · 2026-09-11
- Insider: chances are zero that opt-in Codex data made it into OpenAI training runs — soumitrashukla9 · 2026-09-11
- FrontierMath Tier 4 fully solved: GPT-6 Astra cracks the last problem standing — Jsevillamol · 2026-09-11