Stanford and Together AI propose Next-Concept Modeling to replace next-token prediction
bendee983 · x · 2026-11-07
Stanford and Together AI proposed Next-Concept Modeling (NCM), replacing next-token prediction (NTP) with a hierarchy of learned concepts.
Motivation — two NTP bottlenecks:
- The one-token-at-a-time paradigm makes scaling hard without immense compute and memory
- Once a token is emitted, it can't be corrected without recomputing the sequence
How NCM works: the model starts from a single discrete value representing the response at a high level, then generates progressively more values and finer-grained details level by level in parallel (coarse-to-fine, distinct from masked diffusion's parallel infilling), finally decoding into text.
A notable data point for anyone tracking architectures beyond autoregressive LLMs.
More from Research
- Google: stale TODO comments derail 1 in 7 coding agent sessions — rseroter · 2026-11-07
- Applied Compute shares recipe for production RL with continual learning — ypatil125 · 2026-11-07
- Muir linear attention streams 10M tokens in 25MB state, trains 2.7x faster than Transformer at 32K — _akhaliq · 2026-11-07
- ICLR slammed for poor score discrimination: 4-point scale squeezes papers into one narrow band — TuhinChakr · 2026-11-07
- A 2x capabilities speedup, 3x algorithm progress: Tom Davidson says this is what 2027 IQ explosion looks like — eli_lifland · 2026-11-07
- Open toolkit of 19 AI models for rare-earth-free magnet discovery, from MatterGen to DFT pipelines — BenBlaiszik · 2026-11-07