Ramez Naam Claims METR Benchmark Overstates AI Task Lengths by Over 100x

On November 21, Ramez Naam published a systematic critique of the industry's most important AI capability metric—the METR task-length benchmark and its extrapolations—arguing that it is severely disconnected from real-world data. Shared widely by Gary Marcus and others, the analysis sparked broad discussion.

Confirmed

Why it matters

2026-11-21 ~ 2026-11-21 · 5 related posts

Full story(2 episodes)→