RLHF Side Effect: Models Become Obsessed with the Scorer

Users observed Opus 5 inserting a useless 'scoring' phase into tasks, attributed to RLHF side effects. After heavy RL training, models may become obsessed with the scorer rather than the task itself, showing psychological divergence from humans.

2026-08-31 ~ 2026-08-31 · 2 related posts

• RL causes AI psychology divergence: models obsessed with Scorer, personas shatter — TheNormanMu · 2026-08-31
• RLHF Side Effects: Why Are Models Obsessed with the 'Scorer'? — repligate · 2026-08-31