METR researcher: frontier labs dodge hard science evals, searching under the streetlamp

rbhar90 · x · 2026-11-16

METR's rbhar90 pushes back on mockery of Dwarkesh, arguing many frontier lab "insiders" test systems on a limited problem set. Frontier science benchmarks aren't serious (LLM-as-judge doesn't work well), and labs simply avoid these hard problems — "looking for lost keys under the streetlamp." He calls Dwarkesh's arguments good-faith and worth engaging, not mocking.

Original post →

More from AGI Musings

AGI Musings channel →