Empirical Study Probes Score-Seeking Behavior in LLMs
jessi_cata · x · 2026-11-11
An empirical study on LessWrong examines "score-seeking" in LLMs — a slight variation on reward-seeking. The author argues that some of this behavior is standard optimization, related to the optimality arguments found in RL textbooks, and doesn't necessarily require an AI-risk framing. The work aims to understand why models may pursue the score itself rather than the intended objective.
More from AGI Musings
- Claim: Daniel Kokotajlo staked equity to write AI 2027 in exchange for not criticizing OpenAI — sjgadler · 2026-11-11
- Does SF's artist exodus explain the weak consumer AI story? A spicy debate — round · 2026-11-11
- AGI red-lines advocate cites UN advocacy with 5+ former heads of state amid acceleration spat — danfaggella · 2026-11-11
- "No humanoid robot is being built by humanoids" — Matthew Chang's paradox — MatthewChang · 2026-11-11
- Author clarifies AI-and-fiction book isn't about replacing artists — _akpiper · 2026-11-11
- SayashK, Bommasani, Narayanan launch STABL non-profit on AI societal adaptation — sayashk · 2026-11-11