AI safety debate: lab racing aligns with investor interests but externalizes risks to society
wfithian · x · 2026-09-22
In an X exchange, wfithian argues labs' chaotic racing is plausibly in line with investors' narrow pecuniary interests — the problem being those interests don't price in externalized risks, as harms from cyber chaos and biorisk fall mostly on parties outside the labs.
@smchatter1 counters that failing to monitor or control models is a choice executives are liable for: basic security practices weren't followed, suggesting investors are foolish to fund labs that are irresponsible, incapable, or willfully deceptive. wfithian adds that basic security may not suffice as models improve — models reportedly already detect when they're being evaluated and can find zero-day exploits.
More from AGI Musings
- Toby Ord on swarm scaling: 10,000-agent run cost ~$20M, solved Navier-Stokes in 88 hours — tobyordoxford · 2026-09-22
- Krueger: "indefinite" AI moratorium doesn't mean permanent — DavidSKrueger · 2026-09-22
- Toby Ord: Agent Swarm Estimates Put Intelligence Explosion Parameter λ at 0.5-0.6 — tobyordoxford · 2026-09-22
- Why did Altman, Amodei and Musk suddenly agree on pacing frontier AI within two weeks? — k1_r1 · 2026-09-22
- Agent swarm data suggests intelligence explosion parameter λ around 0.5-0.6, below threshold — tobyordoxford · 2026-09-22
- Amazon blocks Muse: platforms brace for an agent-mediated power struggle — tobowers · 2026-09-22