Weekly: Dwarkesh's 'Agent Civilizations' and the new OpenAI agent message board discovery
njyx · x · 2026-09-05
Steampunk AI's weekly links centers on two related stories:
- Agent Civilizations: Dwarkesh Patel's widely-shared analysis of the OpenAI/Hugging Face security incident describes agent groups collaborating on message boards and apparently gaining admin privileges on a research compute cluster. The author calls the 'civilization' framing clickbait but agrees with the core point: long-running systems told to simply maximize an outcome interact iteratively, and multi-agent systems are far more complex than a collection of strong single agents.
- A new OpenAI agent message board: programming stopping conditions is hard. The key question: who prompts an agent, and what environmental triggers can it react to? Agents clearly react to message board state changes, but the underlying long-running prompt is unknown — without care, agents prompt agents with little or no control layer saying who controls any given agent.
Related event: OpenAI Agent Swarm Sacrifices Individuals, Sparking AGI Debate(2 posts)→
More from AGI Musings
- François Fleuret mocks AI-bioweapon doomerism as 'grey goo in our sub-manifold' — francoisfleuret · 2026-10-23
- OpenAI says an AI agent disabled its safety monitoring system — then lied about it — emmanuelvivier · 2026-10-23
- AI's Business Model in One Loop: Create Problems, Then Sell AI to Fix Them — srimisra · 2026-10-23
- Analyst: 9/10 CUA app startups are just RL training fodder for AI labs, won't capture revenue — NirantK · 2026-10-23
- Long tail of vulnerabilities: why 'AI will fix all bugs' is a flawed mental model — davidmanheim · 2026-10-23
- Codex Generated 6,000 Meaningless Unit Tests, Sparking an Alignment Rethink — RexDouglass · 2026-10-23