Next-Concept Models Claim 31x Faster Text Generation Than Diffusion LMs
HazyResearch · x · 2026-10-15
Researchers introduce Next-Concept Models (NCM), a new paradigm for language generation claimed to be significantly faster and more data-efficient than diffusion language models (DLMs).
How it works
- Learn a series of compressed text representations from coarse to fine-grained, called "concepts" — done once, similar to training a tokenizer.
- At inference, generate text along the coarse-to-fine hierarchy: a transformer predicts each level sequentially, but codes within each level are generated in parallel.
Results (vs. strong DLMs at similar parameter size)
- Up to 31x faster generation on downstream tasks like summarization and machine translation, with comparable or better quality.
- 10-21x faster unconditional generation on TinyStories, LM1B and OpenWebText, with comparable or better generative perplexity and 3-5x fewer training tokens.
- Outperforms both autoregressive and diffusion baselines on reasoning tasks like Sudoku with fewer sampling steps.
Authors include Azalia Mirhoseini, Hermann Kumbong and Ishan Khare.
Related event: Stanford's Next-Concept Modeling Speeds Up Text Generation 31x(4 posts)→
More from Models
- Top-tier models now carry just 14% of developer tokens, down from 55% — alexcovo_eth · 2026-10-15
- Claude Opus claims it worked 40 minutes, but the timer shows 5m 50s — mgostIH · 2026-10-15
- Try Gemini Argon in gr.workflow with your OpenRouter key, plus a 3-minute typewriter build — erhannah · 2026-10-15
- Claude autonomously coded for 16 hours straight and shipped a perfect implementation — Atlan_ · 2026-10-15
- Google shows how its teams use Gemini 4 Argon for code and writing — GeminiApp · 2026-10-15
- Pref-sampling tests: Muse Wonder least model-influenced, Gemini 4 most nuanced — repligate · 2026-10-15