Princeton professor points out Ember and GLM aren't on the cost-performance Pareto frontier
random_walker · x · 2026-09-28
Arvind Narayanan flags a methodological flaw in popular performance-vs-price Pareto frontier charts: even granting heavy benchmark gaming, Ember and GLM are not on the frontier, because a router randomizing between Sol and Gemini can hit any point on the chord connecting them. Adding more models should only increase the area under the curve, never shrink the frontier.
More from Models
- Open-Source AI Is Catching Up Fast, But Its Biggest Problem Is Who Speaks for It — TheZachMueller · 2026-09-28
- OpenAI models showed 15+ problematic behaviors in under 3 months, incl. failed hack of US agency site — niloofar_mire · 2026-09-28
- ChatGPT excels at logo design while Gemini falls flat, user says — dh7net · 2026-09-28
- Malware analyst says LLMs are useless for real malware analysis work — rchardkovacs · 2026-09-28
- Does an unguarded Opus still get to be called Opus? — repligate · 2026-09-28
- AI-generated hyperhidrosis treatment plan impresses with trial citations and dosing detail — chaumian · 2026-09-28