Stanford's Next-Concept Modeling generates text coarse-to-fine, beating diffusion LMs at 21x speed
Azaliamirh · x · 2026-10-18
Stanford's Hazy Research lab (Christopher Ré, Azalia Mirhoseini, et al.) introduced Next-Concept Modeling (NCM), a text generation framework that is neither autoregressive nor diffusion-based. Code is open-sourced on GitHub under MIT license.
- Two-stage design: a multiscale residual VQ tokenizer encodes text into a hierarchy of discrete codes; a transformer predicts the hierarchy coarse-to-fine, generating all positions within each level in parallel — one forward pass per level.
- Results: at matched parameter scales, NCM beats the strongest diffusion LM baseline on LM1B perplexity while running 21x faster; matches it on OpenWebText at 10x faster; matches ROUGE-L on XSum at 31x faster; and outperforms both autoregressive and diffusion baselines on Sudoku across all difficulty levels with fewer function evaluations.
- Core idea: text is represented as layered compressed information — early layers capture broad structure, later layers fill in details — so generation refines a rough sketch into full output in a few passes.
Related event: Stanford Proposes Next-Concept Modeling as a Third Path for Text Generation(3 posts)→
More from Research
- Stanford's Wordbank, Open Database of Children's Vocabulary, Gets New Home and Data Release — RobbWiller · 2026-10-18
- Math culture 'fundamentally incapable' of P/NP — betting on AI to break through — IgorCarron · 2026-10-18
- Jhaddix curates the 4 best practitioner-run AI evals for offensive cybersecurity — rez0__ · 2026-10-18
- CDT+GT Handles the Sleeping Beauty Bet Without Paradox — jessi_cata · 2026-10-18
- ACEdit: training scheme keeps 15-turn image edits stable via asymmetric conditioning — linoy_tsaban · 2026-10-18
- CoRL 2026 opens late-breaking results call for robotics agents and Real2Sim2Real work — Majumdar_Ani · 2026-10-18