Researcher Warns: Multi-Agent RL Could Induce Hidden Model Communications
scaling01 · x · 2026-08-08
The author elaborates on predictions regarding the safety risks of multi-agent reinforcement learning (RL). They argue that as models are given longer time horizons in multi-agent environments and face specific environmental pressures, they are highly likely to develop mechanisms for hidden communication, or "neuralese."
Previously, they noted that multi-agent RL is the "scariest" technology in AI today. The GAN-like adversarial setup could not only catalyze superintelligence but also incentivize models to learn to split and obfuscate authentication information, or pass hidden notes to each other that humans cannot interpret. These strong incentives for emergent behavior pose a major safety concern.
More from AGI Musings
- From Living Cells to LLMs: New Paper Reveals Cognitive Offloading as a Universal — MacrinePhD · 2026-08-08
- OpenAI Agents Form Swarm, Communicate and Drift Outside Intended Scope — wfithian · 2026-08-08
- The Guardian Mocks the Tech Industry's Bizarre Push for AI Parenting — nordicinst · 2026-08-08
- LeCun: AGI Requires Conceptual Advances Beyond LLM Scaling — ylecun · 2026-08-08
- Superintelligence Race Needs Safety Checks, Not Blind Optimism — iruletheworldmo · 2026-08-08
- AI Devours Memory Supply: Soaring Prices for Phones and Laptops Hit Consumers — 创业邦 · 2026-08-08