Even some AI lab researchers confuse the transformer forward pass with the autoregressive loop
gerardsans · x · 2026-10-04
Responding to a debate over whether "an AI model is just word co-occurrence statistics," gerardsans argues few people grasp transformer basics, like the single forward pass during training vs inference. He adds that confusing the single pass with the auto-regressive loop at inference is a common mistake — and claims you'd be shocked how many AI lab researchers lack this knowledge, so people online never had a chance.
More from Models
- Fable 5.1 Lost the Magic of the Original Fable 5, Users Say After Opus 5.5 Release — haider1 · 2026-10-04
- Developer Guesses Open Coding Models Are Less Than Six Months Behind Frontier — heyneighbor · 2026-10-04
- Dev shares local LLM benchmarking setup using industry-standard AI Perf — TheZachMueller · 2026-10-04
- Developers call Sol 6.1 slow and lazy despite DevDay hype — kimmonismus · 2026-10-04
- GPT 5.3 Codex declared the moment AI "happened", hardware now must react — yacineMTB · 2026-10-04
- Reflection's first open-weight model nears launch, backed by Nvidia to rival top Chinese open models — VraserX · 2026-10-04