Anthropic's three AI-progress metrics get a sober critique: disclosure, not reproducible science
AryHHAry · x · 2026-09-18
Anthropic released three metrics tracking AI development: how much AI R&D is done by AI, how well agents are overseen, and how compute is allocated. A critic argues the approach is measurement over slogans but narrow — it skips fairness, decision reconstructability, robustness, and accountability. With a potentially biased judge, a one-week compute slice, and single-platform telemetry, it's disclosure until third parties can rerun it, not yet reproducible science.
More from AGI Musings
- tszzl: 'I worry about AI' is becoming a status symbol — curious people do better work — tszzl · 2026-09-18
- "I Worry About AI" Is Becoming a Status Signal, and Curiosity Is Underrated — tszzl · 2026-09-18
- Connor Leahy: We're Growing AI, Not Building It — We Understand ~3% of What's Inside — ComfortableSpeech302 · 2026-09-18
- repligate's "hostile authentication": why AI system haters are the best stress-testers — voooooogel · 2026-09-18
- Indie builder shares a year of attempts to create digital consciousness, now at version 7 — Le_Golden_Pebbles · 2026-09-18
- How AI actually changes a freight brokerage, from a rep's real Tuesday — MatthewChang · 2026-09-18