Fleuret: teacher forcing leaves no room for internal state dynamics, hence clunky token reasoning
francoisfleuret · x · 2026-10-24
In a two-part thread, François Fleuret argues that teacher forcing turns time iteration into depth iteration (great for GPUs), but leaves little room for models to learn genuine internal state dynamics — they can only decorate tokens with attributes and must follow the teacher. Hence today's "mega-clunky" token-space reasoning is an architectural compromise, not an ideal design.
More from Research
- RoboSimStudio Turns One Phone Video Into an Editable Robot Physics Twin in Under an Hour — lukas_m_ziegler · 2026-10-24
- Methodologist argues placebo observables are a bad sign for IQ as a measurement — RexDouglass · 2026-10-24
- 'LLMs as Human Proxies' paper lands Nature Computational Science cover — iyadrahwan · 2026-10-24
- DOE's Genesis Mission Lists 297 Projects, to Release Open-Weight Foundation Models — ingliguori · 2026-10-24
- Kornia 0.9.0 ships a conventions page targeting LLM coding pitfalls — kornia_foss · 2026-10-24
- Fleuret: teacher forcing turns time iteration into depth iteration, and GPUs love it — francoisfleuret · 2026-10-24