Miles Brundage warns that the AI industry has a growing over-trust problem
Miles_Brundage · x · 2026-07-25
Miles Brundage says the concern is broader than Anthropic alone: the industry has an underappreciated over-trust problem.
- He argues this is likely even more dangerous at companies with faster capability gains and stronger pressure to dogfood their own models.
- The reply frames the issue as an industry-wide safety concern, not just a one-off chart controversy.
- The main point is that over-reliance on model outputs is becoming more dangerous in real-world settings.
Related event: Ex-Researcher Says Anthropic Chart Controversy Stems from Overtrust in AI(4 posts)→
More from AGI Musings
- Misquoted: Anthropic Staff Warned of Double-Digit Extinction Risk by 2030, Not Dismissed It — davidmanheim · 2026-09-11
- Economist Ben Moll: You Can Model Anthropic's 15% AI GDP Growth, But It Won't Happen — sebkrier · 2026-09-11
- Cohere Labs launches interactive tool mapping which tasks of 178 occupations AI can automate — Cohere_Labs · 2026-09-11
- AI researcher on SkyNews flags concerns over inequality, power and criminal misuse — schwarzjn_ · 2026-09-11
- VC compares AI doom rhetoric to pandemic-era fear messaging — StewartalsopIII · 2026-09-11
- Anthropic Insiders: Not Everyone at the Lab Believes in High p(doom) — anpaure · 2026-09-11