Experiments show shared faulty inputs raise all-route agent failures from 4.6% to 7.0%
monkey_spunk_ · reddit · 2026-09-13
The author probes a key risk behind the idea of an agent-run company growing exponentially: when agents review other agents, do shared faulty inputs make models fail together?
- A first study weakened the concern that companies sharing foundation models make identical mistakes: after adjusting for task difficulty, residual failure overlap was near zero, and changing the model endpoint reduced overlap more than changing the evidence source.
- A follow-up used 8 fixed model/deployment routes on synthetic records with one recoverable wrong field. When every route received the identical corrupted version, the average exact-repair failure rate barely moved (59.2% → 59.9%), but cases where all eight routes failed rose from 77 to 117 (4.59% → 6.98%).
- Takeaway: reviewer count and individual accuracy alone won't reveal how often a mistake passes every check; agent-run operations must measure shared failures directly. Results are preliminary and not peer reviewed.
More from AGI Musings
- Dario's 'AI accelerating AI' claim contradicted by Anthropic's own AECI benchmark — eli_lifland · 2026-09-13
- Altman: AI went from grade-school math to a Millennium Prize problem in 3 summers — rohanpaul_ai · 2026-09-13
- Blogger mocks doomer logic: if AI beats all humans, regulation is futile — Kyrannio · 2026-09-13
- Public pushes back on 'AI billionaire warns AI may kill you' PR strategy — venturetwins · 2026-09-13
- X Debate: Utilitarianism's Global Max Isn't Fully Automated Human Luxury Communism — jessi_cata · 2026-09-13
- Nvidia Would Tank If It Adopted UALink — Why the Regulatory Capture Argument Falls Apart — itsclivetime · 2026-09-13