Dietterich: LLMs Go Beyond Next-Token Prediction

ML researcher Thomas Dietterich pushed back on claims that LLMs merely predict the next word, arguing that SFT and RL have moved models toward predicting correct multi-token sequences, and that pretraining itself forces abstract generalization beyond parroting.

2026-09-25 ~ 2026-09-25 · 2 related posts

• Tom Dietterich: SFT and RL go beyond next-token prediction in LLMs — tdietterich · 2026-09-25
• Dietterich: pretraining's next-token task already forces models to abstract and generalize — tdietterich · 2026-09-25