FULL STORY

METR's Anthropic Assessment: From Report to Redaction Dispute

METR's first independent assessment using Anthropic's internal data found AI roughly doubling the pace of the lab's own R&D. The fallout—redactions, a cancelled compute-eval pilot, and claims of undisclosed frontier models—has drawn scrutiny from safety researchers.

2026-11-06 ~ 2026-11-06 · 6 episodes · 58 posts

Episode 1 · METR Assessment of Anthropic's AI-Driven R&D Shakes Community, Pulls RSI Timelines Forward (2026-11-06, 47 posts)

METR has released the first independent risk assessments of whether AI is accelerating AI R&D, drawing on internal data from a frontier lab. Its team was granted access to Anthropic's internal information starting August 2025, concluding that as of October 2026, the rate of capability growth on well-defined tasks has reached roughly 2x the pace observed between March 2024 and October 2025. This is the most direct first-hand benchmark evidence that AI capabilities are accelerating—while heavy redactions in the public report have raised transparency concerns.

Confirmed

  • METR gained access to Anthropic internal information from August 2025 to evaluate whether AI is accelerating AI R&D, and has now published a report summary (m3, m4, m6)
  • Core conclusion: as of October 2026, model capability growth on well-defined tasks is roughly 2x the rate of March 2024–October 2025, with large uncertainty and a plausible range of 1.3x–3x (m5, m10)
  • METR's AECI metric analysis shows that since October 2025, Anthropic's model scores have grown about 1.9x faster than earlier (95% confidence interval 1.5–2.8) (m8)
  • Anthropic's internal Model 4 finished training in October 2026 and scored 178.6 on AECI; @scaling01 said this validates an earlier prediction (m1, m7)
  • METR judged that Anthropic has likely triggered the 2x R&D acceleration threshold in its Responsible Scaling Policy (RSP), possibly as high as 3x, and guesses that roughly 80% of the current acceleration comes from AI (m9)
  • METR submitted a roughly 100-page full evaluation to Anthropic; under their agreement Anthropic had the right to redact non-public information, saying the redactions removed confidential details that "could accelerate other AI developers"; METR said that while it could freely share the top-level risk conclusions, most of the evidence it considered most informative was removed from the public version (m2, m4, m11)
  • Anthropic co-founder and safety lead Sam McCandlish published a long post responding to the report for the first time, previewing more safety progress updates in the coming months (m6)

Unconfirmed

  • The precise acceleration factor remains highly uncertain: @daveholtz noted this is important data, but given the statistical uncertainty, it's hard to conclude the 2x threshold was actually triggered (m9); the conclusion relayed by @harrisedouard also emphasized this is a subjective risk judgment
  • The "80% of acceleration from AI" figure is a speculative METR estimate

Why it matters

  • This is among the first independent evaluations to quantify "AI self-acceleration" directly using a frontier lab's internal data, providing measurable benchmark evidence for intelligence explosion concerns
  • The report suggests the growth rate may have hit a key threshold in the RSP framework, directly bearing on the safety measures Anthropic must take
  • However, METR has not concluded that full Recursive Self-Improvement (RSI) has been achieved (m10)—the acceleration remains at a "partial" stage

27 more related posts →

Episode 2 · Anthropic's Internal Models Reportedly Surpass Mythos 5 (2026-11-06, 2 posts)

Leaks suggest Anthropic has iterated at least three flagship checkpoints beyond Mythos 5, with employees reporting 12x speedups, and a METR chart revealed an unreleased Model 4 scoring 178.6 on the AECI index.

Episode 3 · Anthropic Faces Backlash Over Heavy Redactions of METR Risk Report (2026-11-06, 3 posts)

METR said most of the key evidence in its roughly 100-page risk assessment of Anthropic was redacted from the public version. Anthropic's co-founder acknowledged the cuts, citing safety and commercial sensitivity, while critics including Gary Marcus blasted the move.

Episode 4 · Report: Anthropic's Next Model Finished Training, Tops All METR Benchmarks (2026-11-06, 2 posts)

A leaked METR private evaluation suggests Anthropic has completed training a unreleased model (codename Model 4) that scored highest among all models METR has analyzed, exceeding 50% on all 15 benchmarks, with release expected early next year.

Episode 5 · Insider: 7 of Anthropic's Last 8 Frontier Models Never Publicly Benchmarked (2026-11-06, 2 posts)

An insider claim reveals that 7 of Anthropic's last 8 frontier models were never publicly benchmarked, suggesting public evaluations may understate the accelerating pace of model progress.

Episode 6 · METR Says Anthropic Canceled Compute Assessment Pilot, Sparking Safety Criticism (2026-11-06, 2 posts)

METR revealed that Anthropic canceled a planned compute allocation assessment pilot, drawing public criticism from AI safety researchers, including former OpenAI researcher Daniel Kokotajlo, who accused Anthropic of heavily redacting safety evaluation information under secrecy claims.