LLaDA 2.2’s block-diffusion MoE edits its own outputs during training and inference
omarsar0 · x · 2026-07-26
This follow-up explains the block-diffusion MoE design behind LLaDA 2.2.
- The diagram shows a three-stage pipeline: pretraining with Levenshtein editing, supervised fine-tuning on long-context and agent data, then RL with Levenshtein editing.
- It also shows the inference side: block routing selects a subset of experts per block to keep routing efficient.
- The core idea is to let the model edit its own spans instead of locking in early token mistakes.
More from Models
- Kimi K2.8 Preview rolls out: near-K3 coding performance, 1M context for all tiers — teortaxesTex · 2026-09-11
- Looking for a classifier of software engineering task shapes to pick models per task — StewartalsopIII · 2026-09-11
- DeepSeek V4 Pro API to continue after Sept 2026, billing unchanged — teortaxesTex · 2026-09-11
- DeepSeek V4.1 Flash Hits 98% of GPT-6 Astra's Score at 1.4% of the Cost in Third-Party Benchmark — ayushtweetshere · 2026-09-11
- TheZvi Polls: Has Your Coding Model Choice Changed Since Fable 5.1 and Astra? — TheZvi · 2026-09-11
- antirez Weighs In on Anthropic Banning Minors From Using Claude — antirez · 2026-09-11