LLaDA 2.2’s block-diffusion MoE edits its own outputs during training and inference
omarsar0 · x · 2026-07-26
This follow-up explains the block-diffusion MoE design behind LLaDA 2.2.
- The diagram shows a three-stage pipeline: pretraining with Levenshtein editing, supervised fine-tuning on long-context and agent data, then RL with Levenshtein editing.
- It also shows the inference side: block routing selects a subset of experts per block to keep routing efficient.
- The core idea is to let the model edit its own spans instead of locking in early token mistakes.
More from Models
- Unverified: GPT-6.1 Terra reportedly released, leaked chart shows top score per dollar on Agents' Last Exam — ZeroStateReflex · 2026-10-27
- GPT-6.1 Terra cuts factual error rate from 22.3% to 4.3%, an ~81% drop — haider1 · 2026-10-27
- GPT-6.1 Terra lands: early users praise speed, value and subagent performance — BenBajarin · 2026-10-27
- Leaked GPT-6.1 Terra benchmarks: cheap but 'boring', still far behind Astra and Claude — Angaisb_ · 2026-10-27
- Artificial Analysis Ships Intelligence Index v4.3.2 With GPT-6.1 Terra Comparison — ArtificialAnlys · 2026-10-27
- GPT-6.1 Terra cuts hallucination rate by 50 points while lifting accuracy 2 points — ArtificialAnlys · 2026-10-27