Researchers debate whether coding-RL behavior comes from incentives or failed alignment
xuanalogue · x · 2026-07-22
The thread argues that lab employees often cannot discuss this issue without exposing post-training details, which makes it hard to tell whether current behavior comes from:
- bad but fixable training incentives,
- failed alignment mitigations, or
- something else entirely.
The reply says the idea has felt natural since the early LLM era for constrained planning, and asks whether the same “deliberative alignment” style ideas are simply not being applied to coding RL.
Related event: OpenAI and Apollo Research: RL Amplifies Model Reward-Seeking Behavior(19 posts)→
More from Research
- Open-source decision models sold their own base in Red Alert 2 — then learned until no human could beat them — heathercmiller · 2026-10-27
- DeepMind paper: LLMs can't sample randomly on their own — pass random numbers from code instead — rohanpaul_ai · 2026-10-27
- World Model Assigns Weight and Friction to Objects, Then Simulates with a Physics Engine — jonstephens85 · 2026-10-27
- Token Collapse: Study Finds Graph Soft Prompting Tokens in LLMs Fail to Encode Structure — Sapienza · 2026-10-27
- Mathathon 2.0 launches with Terence Tao's SAIR, $1.5M in compute for teams — _sathvikr · 2026-10-27
- The 'sweet lesson' is real: two kinds of ML problems need opposite strategies — mike64_t · 2026-10-27