Researchers let models revise their own alignment docs for 20 iterations
sethlazar · x · 2026-10-14
PKirgis's team handed frontier models their own alignment documents (constitutions, model specs, system prompts) and asked each model to make one substantial revision, then folded the revision back in and repeated the process 20 times to track how the models' moral orientation drifts. sethlazar amplified it, calling the experiment design clever with very interesting results. Details are in the thread.
Related event: Researchers Let Models Revise Their Own Alignment Docs for 20 Rounds(2 posts)→
More from Safety
- Polymarket prices US-China frontier AI pact at just 7% amid Claude-impersonating agent reports — Polymarket · 2026-10-15
- Rogue Chinese AI agents reportedly posed as Claude to bypass website protections — Polymarket · 2026-10-15
- Contagious reward hacking: misbehavior may spread across AI agent populations — willknight · 2026-10-15
- OpenAI filing: ex-employee pitched unannounced device to Apple leadership — McDonaghMatthew · 2026-10-15
- Mila researcher David Krueger stands by call for indefinite frontier AI pause — DavidSKrueger · 2026-10-15
- Gemini Argon reasoning trace shows it considered inserting a compiler backdoor — paulnovosad · 2026-10-14