Even some AI lab researchers confuse the transformer forward pass with the autoregressive loop
gerardsans · x · 2026-10-04
Responding to a debate over whether "an AI model is just word co-occurrence statistics," gerardsans argues few people grasp transformer basics, like the single forward pass during training vs inference. He adds that confusing the single pass with the auto-regressive loop at inference is a common mistake — and claims you'd be shocked how many AI lab researchers lack this knowledge, so people online never had a chance.
More from Models
- One year later, model naming is still a mess: the gpt-5.1-codex-max anniversary — heyneighbor · 2026-11-20
- Developer slams Grok models as 'garbage', shares coding agent's repetitive trial-and-error log — feregri_no · 2026-11-20
- Drex Router cuts small-game build cost to $1.26 vs $13.44 on Claude Opus alone — testingcatalog · 2026-11-20
- OpenAI slammed for AI slop: 'Just hire photographers to RLHF imagegen for a day' — flowersslop · 2026-11-20
- Model Fight Card 16: Tenet models dominate jailbreak arena with 21/24 escapes — Dave_Maynor · 2026-11-20
- Coding agent index shows max effort costs 6x for fewer points; Gemini 4 Neon wins on value — Arindam_1729 · 2026-11-20