Pareto Frontiers Should Be Convex Hulls: Princeton Team Slams Flawed Model Benchmark Charts

random_walker · x · 2026-09-28

Arvind Narayanan's team points out that popular model comparison charts misuse Pareto frontiers: Ember and GLM are not on the frontier, because a router can switch between Sol and Gemini to hit any cost-accuracy point on the chord connecting them — adding models should only grow the area under the curve, never shrink it.

They made these points two years ago in AI Agents That Matter (arXiv:2407.01502), one of the first papers to call for Pareto curves over accuracy-only tables. Key arguments:

Related event: Princeton Professor Flags Misuse of Pareto Frontier in AI Model Comparison Charts(2 posts)→

Original post →

More from coding & agent

coding & agent channel →