Hamel Husain: AI benchmark firms are the new credit rating agencies
jxnlco · x · 2026-11-02
Hamel Husain argues that AI benchmark companies are structurally the same business as Gartner vendor rankings, the credit rating agencies behind mortgage-backed securities, and US News college rankings: paid parties shaping the rankings. "Enough people haven't caught onto it yet, but they will." jxnlco replied asking whether METR is included.
Related event: Hamel Husain Slams Public AI Benchmarks as Rating-Agency-Style Business(4 posts)→
More from Models
- Grok Bot pricing debate: users say free tier makes no sense, back gifting instead — RachelVT42 · 2026-11-02
- Grok users hit weekly limits early as OpenAI doubles Pro caps — how do teams plan? — Entire_Struggle_2236 · 2026-11-02
- PyTorch interview question: model.eval() vs torch.no_grad() explained — cneuralnetwork · 2026-11-02
- Reflection to host Reddit AMA on Nov 3 about Beam, its first open-weight model — Aggressive-Sign-8167 · 2026-11-02
- Cursor Pro+ user burns through all quotas, says forced to upgrade to $200 Ultra plan — pramodk73 · 2026-11-02
- User regrets $500 OpenAI plan: 'ultra fast' is actually buggy computer use — imjustnewatai · 2026-11-02