Knowledge Distillation was invented by Rich Caruana, Hinton just popularized it
khademinori · x · 2026-11-03
Weighing in on whether an agent reimplementing a simple mech interp idea counts as novel research, the author argues that popularization is part of science: Rich Caruana invented Knowledge Distillation, while Geoffrey Hinton made it famous. Quoted, Yavuz Bakman notes he used Claude Opus 5-5 two weeks ago to suppress an undesired feature in a model for a take-home research interview, and says the Qwen experiments in question are no more impressive — with today's wealth of interpretability research and open-source repos, an agent doing simple mech interp shouldn't surprise anyone.
More from Fun
- Theo: LLMs got good at everything except the one thing they were built for — text — film_girl · 2026-11-03
- 'Social media is rotting our brains' — also me, happily watching AI dancing bears — kyliebytes · 2026-11-03
- 'You'll get to watch my music videos': AI circle roasts Core Auto recruiting pitch — iScienceLuvr · 2026-11-03
- Railway's "oops, sorry" post sparks debate over agentic security probing — seatedro · 2026-11-03
- Claude-generated SVG of astronaut on pelican bike nails real road-bike geometry — ChrisGPT · 2026-11-03
- Meme: 'You're hiding 128 GB of RAM under the floorboards, are you not?' — cneuralnetwork · 2026-11-03