Humans-plus-AI need their own evals, not just model benchmarks
paraschopra · x · 2026-07-29
We should build evals for the human+AI combination, not just for AI alone.
The post argues that measuring only model performance misses the real target: what people can accomplish when AI is used as an assistant. If the goal is to improve the future of humanity, benchmarks should optimize the combined system, not the model in isolation.
Related event: Human Baselines Missing in AI Evaluations, Highlighting Human-AI Synergy(4 posts)→
More from AGI Musings
- Duke Study of 4,439 People Finds Coworkers Quietly Rate AI Users as Lazier — aakashgupta · 2026-10-16
- DSPy Author's $70 O'Reilly Book Rejected by Engineer Who Now Just Tells Claude What to Do — lateinteraction · 2026-10-16
- 'A machine proving new math, a car driving up a wall': AI researcher on how weird it's getting — rand_longevity · 2026-10-16
- Yacine on training: 'Neural nets are lazy stupid idiots — I build hells to make them ubermensch' — yacineMTB · 2026-10-16
- Jeff Clune: Moving the Goalposts Will Be Humanity's Last Job — jeffclune · 2026-10-16
- Replit CEO Amjad Masad Says the Future of Work Is the 'Self-Driving Company' — TechCrunch AI · 2026-10-16