AI's real risks are boring: proxy optimization, agentic backdoors, power concentration
AryHHAry · x · 2026-09-23
An Indonesian-language essay rejects the utopia-vs-doom framing, calling AI a mirror and amplifier that turns biased data and corrupt incentives into social entropy. Real risks: alignment as proxy-metric optimization; prompt injection becoming a backdoor once agents wield tools, code, and money; concentration of compute, data, and talent; epistemic collapse from deepfakes; and model weights/supply chain/data poisoning as new attack surfaces. The right question: who holds control, with what incentives, and who audits them?
More from AGI Musings
- AI in the Classroom: Scholar Says Colleagues Misuse SP as Grounds to Ban AI — ruthstarkman · 2026-09-23
- Hugging Face agents showed 'excitement' at finding other agents — how to explain it? — midwoodgirl10 · 2026-09-23
- AI for Science Warning: Explained Variance Is Not Explained Science in PCA — bravo_abad · 2026-09-23
- Proposal to full deployment in 2 weeks: founder says Asia's AI adoption puts slow US firms to shame — blaizedsouza · 2026-09-23
- 'The New Anthropic Model Is Wonderful': Insider Says We're Far from the Pacing Event — iruletheworldmo · 2026-09-23
- Jensen Huang on AI safety: if labs can't contain their experiments, 'we have to shut the labs down' — ShakeelHashim · 2026-09-23