Evals are vanishing from model cards since ChatGPT went mainstream
evijit · x · 2026-09-14
The author points to their team's prior work examining which evaluations stopped being disclosed in model/system cards after ChatGPT's mainstream commercialization — and the picture is bleak. Full analysis is in their ICML paper covering 186 first-party reports and 248 third-party evaluation sources.
Related event: ICML Paper Finds AI Social Impact Evaluations Shrinking(3 posts)→
More from Research
- AIR: Effective-Learning-Rate Scheduler Beats Cosine on 128B-Token MoE Pretraining — stochasticchasm · 2026-10-30
- CMU's Gym-Anything Auto-Builds 200+ Agent Environments from Any Software, Selected by GDP — wellecks · 2026-10-30
- NVIDIA researcher: train robot policies with privileged sim info, improving Pi 0.5 — wellecks · 2026-10-30
- Leveraging Simulator Privileged Information: Three Methods to Improve Robot Models — wellecks · 2026-10-30
- CMU Auto-Builds 25 Robot Simulations with Agents; Frontier Model Solves 0 of 25 Test Tasks — wellecks · 2026-10-30
- DeepMind's f_charton shares his take on the current state of AI and mathematics in Sorbonne talk — f_charton · 2026-10-30