Yoav Goldberg publishes LLM explainer for an informed general audience
yoavgo · x · 2026-08-26
Noted NLP researcher Yoav Goldberg published an explainer of how LLMs actually work, targeted at an informed general audience — something he says he had yet to see done properly. Co-created with Tim and inspired by CIFAR LMB discussions, it was refined with Brett Mensh and will be part of an upcoming manuscript; it's still work in progress and open to feedback.
One informative detail he highlights: at the end of pre-training, a model completes sentences but doesn't perform tasks — "what color is the sky?" gets continued as "I asked my daughter when..." rather than answered with "blue". Task behavior emerges only in later training stages.
More from Research
- EmbodiedEvo tops RoboDojo at 38.95% success by letting robots inherit failed practice — thetripathi58 · 2026-10-15
- 797GB of n-gram counts in CPU RAM let 1B models match 57% more compute — PangWeiKoh · 2026-10-15
- Google's HARNESS² evolves agent harnesses without verifiers, up to +11.5 points on WorkBuddy-Bench — dair_ai · 2026-10-15
- Subbarao Kambhampati to dissect the 'mythos of LLM thinking tokens' at UMN — rao2z · 2026-10-15
- How 3D Gaussian Splatting gives robots spatial memory of what's out of view — TheTuringPost · 2026-10-15
- 10 Health Science Papers Written With muse Models in Under 4 Weeks — armand_ruiz · 2026-10-15