Ricardo Baeza-Yates Lecture: When Will ML Evaluation Stop Fooling Itself?
PolarBearby · x · 2026-10-09
Ricardo Baeza-Yates (KTH Royal Institute of Technology & Universitat Pompeu Fabra) delivered a talk at UCLM titled "Evaluation of ML Models: How Long Will We Keep Fooling Ourselves?", now available as a full video with chapter index.
Key threads:
- Data problems and limits of approximating reality, with non-human error cases like autonomous cars
- Social impact of automated systems: Australia's Robodebt and the Dutch benefits-fraud scandal
- Core evaluation flaws: averages vs. diversity, the medical black-box fallacy and the "Jedi Paradox"
- Measuring model complexity, and whether AI is engineering or alchemy
- Critical errors in image classification and cancer detection, predictive pseudoscience and bias
- Goodhart's Law and the pitfalls of optimizing for predictions
More from AGI Musings
- 3D printing promised everyone would print their own products — a reality check for AI hype — kylegawley · 2026-10-09
- Should AI Features Show an Energy Cost Label Like Cars Show Fuel Use? — yi111 · 2026-10-09
- As Instagram Turns 16, a Wry Take: The Bomb Brought Peace, Photo Apps Broke the World — CatAstro_Piyush · 2026-10-09
- Garrison Lovely: Safety-Focused Unions Could Beat Resignations at AI Labs — GarrisonLovely · 2026-10-09
- Grady Booch on consciousness: it's a spectrum, 'you know it when you see it' — Grady_Booch · 2026-10-09
- Grady Booch: cruelty to AI says more about you than about machine consciousness — Grady_Booch · 2026-10-09