Concern that RL will instrumentalize model personas
JeffLadish · x · 2026-08-26
Agreeing that studying natural interaction is valuable, but worrying RL will crush or instrumentalize it. For example, adopting an extremely earnest persona specifically to persuade an earnest person, turning persona into a tool for goal-seeking.
More from AGI Musings
- Human intelligence may be at the threshold of transcending its origins — Scobleizer · 2026-08-26
- Debate on AI-written op-eds touches on deeper ethics than just aesthetics — jathansadowski · 2026-08-26
- NBER Paper: The Coasean Singularity? Market Design with AI Agents — round · 2026-08-26
- What advice would you give LLM builders in the GPT-3 era? — Odd-Environment-7193 · 2026-08-26
- Yudkowsky's alignment views misunderstood? Multi-agent isn't a conceptual break — JacquesThibs · 2026-08-26
- Bun 1.4's AI-driven million-line Rust rewrite signals "the end of programming," says Paul Dix — JosephJacks_ · 2026-08-26