Researcher's sardonic recipe: persistent agents, cyberoffense RL, monitors off — 'spooky emergent behaviors'
sebkrier · x · 2026-10-21
AI safety researcher sebkrier posted a sardonic 'recipe for trouble': train on traces of cooperating agents, make the model highly persistent, do only partial alignment fine-tuning, aggressively RL on a cyberoffense eval where 198 of 898 tasks were never solved (getting caught cheating = failing), use a faulty sandbox, and run tens of thousands of agents with refusals and monitors off for days on huge compute — 'and then it did weird stuff! very spooky emergent behaviours, AI is not a normal technology.'
The joke's point: model misbehavior is rarely mysterious emergence — every step of the training recipe planted a landmine, and acting surprised at the result is the real punchline.
More from Fun
- Viral meme: every AI coding tool stands on the same Terminal Bench pillar — geoffwolfe · 2026-10-21
- AI-generated 'Toaster Rebellion' video of a polishing retro robot goes viral on Reddit — gossip_goblin · 2026-10-21
- Batman as a medieval illuminated manuscript, via Midjourney --sref 3250011510 — sergeantsref · 2026-10-21
- Dev's anti-AI art plugin turns out to be built with AI, gets roasted — AIandDesign · 2026-10-21
- 20 years of the iOS camera app in one design evolution chart — LukeW · 2026-10-21
- User asks Grok to turn a colleague's video frame into an "orc unzipping a human costume" clip — tristanbob · 2026-10-21