Fleuret: teacher forcing turns time iteration into depth iteration, and GPUs love it
francoisfleuret · x · 2026-10-24
François Fleuret highlights an elegant property of transformers: thanks to teacher forcing, iterating a medium-size state over time is transformed into iterating a gigantic state over depth — which GPUs are happy about.
More from Research
- Methodologist argues placebo observables are a bad sign for IQ as a measurement — RexDouglass · 2026-10-24
- 'LLMs as Human Proxies' paper lands Nature Computational Science cover — iyadrahwan · 2026-10-24
- DOE's Genesis Mission Lists 297 Projects, to Release Open-Weight Foundation Models — ingliguori · 2026-10-24
- Kornia 0.9.0 ships a conventions page targeting LLM coding pitfalls — kornia_foss · 2026-10-24
- Domingos: An attention head is just a fast-weight MLP — pmddomingos · 2026-10-24
- Ross Taylor: studying LLM behavior will increasingly resemble social science — rosstaylor90 · 2026-10-24