Anthropic's three AI-progress metrics get a sober critique: disclosure, not reproducible science
AryHHAry · x · 2026-09-18
Anthropic released three metrics tracking AI development: how much AI R&D is done by AI, how well agents are overseen, and how compute is allocated. A critic argues the approach is measurement over slogans but narrow — it skips fairness, decision reconstructability, robustness, and accountability. With a potentially biased judge, a one-week compute slice, and single-platform telemetry, it's disclosure until third parties can rerun it, not yet reproducible science.
Related event: Anthropic Discloses Internal Metrics: Claude Now Drives 26% of Its R&D(41 posts)→
More from AGI Musings
- Avoiding AI brain rot: Miles Cranmer's lecture on learning with AI — _ScottCondron · 2026-11-02
- Yacine mocks "autonomous research" hype as Naam argues RSI loop needs 5-10x boost to run away — yacineMTB · 2026-11-02
- Vitalik's "post-AI cryptography": pseudonyms are dead, per-action anonymity is in — devanshmehta · 2026-11-02
- Can law be formalized as deterministic rules? A French thread argues LLMs unlock the soft cases — IgorCarron · 2026-11-02
- Nobel economist William Nordhaus dies; essays revisit his growth, tech and climate work — Afinetheorem · 2026-11-02
- dhh: agents' great gift is routing around every gatekeeper in software — Stack Overflow proved it — IgorCarron · 2026-11-02