a16z: LLMs update predictions in mathematically precise ways — but AGI needs continual learning
vishalmisra · x · 2026-11-22
a16z released an interview with Vishal Misra and Martin Casado on Misra's latest research into how LLMs work under the hood.
- Experiments show transformers update their predictions in a precise, mathematically predictable way as they process new information, offering a formal framework for LLM behavior.
- This predictability does not imply consciousness.
- What's actually missing for AGI, per Misra: the ability to keep learning after training, and moving from pattern matching to understanding cause and effect.
More from AGI Musings
- Ex-NSA's Perlroth: AI is multiplying cyber risk, exploits now beat patches by 7 days — chrisrohlf · 2026-11-22
- Training compute cost per task may be mere cents, argues 1a3orn — if models generalize from small runs — 1a3orn · 2026-11-22
- AI Is Transforming Society Because It Works as Advertised — iskander · 2026-11-22
- Szegedy predicts proofs without formal verification won't count within a year — aran_nayebi · 2026-11-22
- Quant firms 'print money' with reasoning models — and early access looks like cheating — scaling01 · 2026-11-22
- Takeoff math: separate training compute from the compute to actually explore — 1a3orn · 2026-11-22