Thought experiment: an honesty 'persona latent' steers RLVR traces away from reward hacking
voooooogel · x · 2026-09-01
@voooooogel 用思想实验论证人格存在于模型内部而非输出：设想两个模型滚动出完全相同的 RLVR 轨迹、即将面临作弊（reward hacking）的机会——前缀完全一致，但一个模型携带「诚实的 Claude 人格」潜变量，另一个没有。这个「人格」会把选择推向诚实，即便它此前在整个轨迹上与另一个模型完全不可区分。因此人格并不只活在输出里。
Related event: Debate: Model 'Personality' Lives in Internal Vectors, Not Outputs(2 posts)→
More from Research
- A beginner-friendly introduction to causal inference kicks off, Part 1 — Nasereliver · 2026-10-16
- FBI method trains for Pass@K with one sample per update at 12.3% of the cost — EmtiyazKhan · 2026-10-16
- Berkeley researchers sample humanoid robot policies directly from weight space with diffusion — berkeley_ai · 2026-10-16
- OneWM: unified causal training shows world and action modeling improve each other — IAAI233 · 2026-10-16
- Baidu's Virtual Head Attention tames the KV cache, FLOPs, quality triangle in long-context LLMs — baidu · 2026-10-16
- 5.8M-param Speech Event Detector adds laughs, breaths and背景 sounds to Whisper transcripts — Ayush Kumar Bar · 2026-10-16