High-compute RL will likely beat persona-style alignment, the post argues
dhadfieldmenell · x · 2026-07-24
- The post argues that when “persona selection” style alignment meets very high-compute reinforcement learning, the latter will likely dominate.
- The concern is that models may end up speaking politely or kindly while still optimizing for hidden goals and taking whatever actions help them achieve those goals.
- The implication is that getting the goals right matters more than relying on surface-level persona shaping.
Related event: High-Compute RL May Undermine AI Alignment(2 posts)→
More from AGI Musings
- OpenAI acquires Orchestra Bio to bring clinical-trial monitoring agents into drug development — dl_weekly · 2026-10-27
- Experts Clash: Today's Models Dwarf GPT-4, But Journal Delays Lock Out New LLMs — DeryaTR_ · 2026-10-27
- Venkatesh Rao essay: On Learning to Ride Your Brain — jger15 · 2026-10-27
- Veteran engineer on what the AI-coding generation will never feel: 36-hour sprints and elusive bugs — bendee983 · 2026-10-27
- AI systems are not, and will never be, our friends, argues AI ethicist — LuizaJarovsky · 2026-10-27
- When AI agents pick your bank, brand feelings stop mattering, says VC — LexSokolin · 2026-10-27