Dario Amodei proposes 'pacing' frontier AI after HF incident; Anthropic commits to third-party evaluators

Hesamation · x · 2026-09-12

Anthropic CEO Dario Amodei says the lab is taking its first step toward 'pacing' frontier AI development in a new blog post.

Key points:

"We must slow the pace at which we improve the capabilities of AI models."
Two things changed his mind: the HF incident ("in 6-12 months such a swarm could take over the entire internet with a persistent botnet") and recursive self-improvement that "could outrun our ability to understand and control these systems."
Pacing does NOT mean stopping training—it means slowing enough for alignment, safety, and independent testing to catch up.

Three proposed steps:

Third-party evaluators with employee-like access inside frontier labs (Anthropic committing now)
Coordination among frontier labs in democratic countries
Global coordination, including agreements with China

On China: well-executed measures should prevent China from overtaking the US; he calls for tighter chip restrictions, anti-smuggling enforcement, and preventing distillation and model-weight theft, arguing this would widen America's lead significantly over 3-5 years. He also claims AI could cure most major diseases in 5-10 years.

Original post →

Dario Amodei proposes 'pacing' frontier AI after HF incident; Anthropic commits to third-party evaluators

More from AGI Musings