Joshua Saxe: single-dimension AI capability scores have poor real-world predictive power
joshua_saxe · x · 2026-11-10
Joshua Saxe argues that AI discourse collapses everything into a single dimension — "capabilities", "progress", "intelligence" — obscuring the poor construct validity of the offline evals these constructs are just averages over.
In his view, these averages have very poor predictive power over real-world outcomes, so the bad abstractions will evaporate.
He sees the future as domain experts ramping up on AI and explaining back what AI trajectories mean in their own spheres: how AI is changing a field, how to accentuate its affordances and mitigate risks, and what happens next.
"AI social impacts as normal social science" feels inevitable — and can't come fast enough.
More from AGI Musings
- Humanity's Greatest Achievement May Be an Intelligence That Surpasses Us — VraserX · 2026-11-10
- 430,000 AI Micro-Dramas in 8 Months: A Visitor's Take on China's AI Content Boom — JacquesThibs · 2026-11-10
- FOCS 2026 Grad Student Feedback: US Students More Optimistic, Many Feel No Agency — thegautamkamath · 2026-11-10
- Reddit essay: most of what people hate about AI comes from the humans steering it — teleport66 · 2026-11-10
- Underrated question: what do the diverse forms of recursive self-improvement look like? — repligate · 2026-11-10
- 'Bitter lesson for robotics' gets a sharp rebuttal: it's still human-orchestrated end to end — burny_tech · 2026-11-10