Andrew Dai: pre-training + fine-tuning was born from a forgotten checkpoint bug
ziv_ravid · x · 2026-09-27
- New episode of The Information Bottleneck features Andrew Dai, co-founder and CEO of Elorian AI, on vision models. Dai spent 14 years at Google.
- Dai co-authored the 2015 paper introducing pre-training and fine-tuning for language models — a result he says came from a bug: he forgot to update a checkpoint directory and accidentally fine-tuned a language model.
- The episode also covers why Google fell behind early in the LLM race, and why models still can't count objects in a photo. Dai argues much reasoning is visual, which is what Elorian is building for.
More from Fun
- AI agent monitors Delta seats overnight and auto-switches user to a window seat — unixterminal · 2026-11-11
- "QT your Geisha" meme takes over X with stunning porcelain-style AI portraits — LudovicCreator · 2026-11-11
- Developer locked out of his MacBook by a swarm of AI agents running mid-task — theteknosaur · 2026-11-11
- Yacine shares his daily grind: up at 6:50 am, agents checked overnight, work till 3 pm — yacineMTB · 2026-11-11
- 'Adam can now do CAE': AI agent showcase in engineering simulation draws buzz — burhop · 2026-11-11
- Anthropic engineer's Fable 5.2 one-shot 'AAA-level' game sparks AI-renaissance debate — RileyRalmuto · 2026-11-11