AlexNet Should Never Have Worked: Simon Prince on deep learning's lucky break
Machine Learning Street Talk · youtube · 2026-11-12
Simon Prince, author of Understanding Deep Learning (MIT Press, 2023), explains on Machine Learning Street Talk why AlexNet 'should never have worked': 60 million parameters — orders of magnitude beyond contemporary vision models — a 150,000-dimensional input space, trained from random weights with noisy gradient descent, yet it fit ImageNet. 'Jaw-dropping, really.' This short is an excerpt from the full conversation.
More from Research
- iamtrask: weight columns are concept detectors, and labs profit from not auditing them — iamtrask · 2026-11-12
- Qwen 3.8 27B on a laptop can read model weights, replicating the Weight-Space Programming paper — dl_weekly · 2026-11-12
- Trask: Transformer weight columns are concept detectors, but many concepts remain missing — iamtrask · 2026-11-12
- Dense rewards lose to standard GRPO at matched compute, arXiv paper finds — burny_tech · 2026-11-12
- Wayve unveils Learned Safety Envelope: end-to-end trained live risk prediction for AI driving — alexgkendall · 2026-11-12
- HipKittens turns one: breaking AMD's 'CUDA moat' for a multi-silicon future — simran_s_arora · 2026-11-12