5,664 single-word teacher answers lift student +5.34pp on HumanEval+ without any code
_akhaliq · x · 2026-09-30
- The paper Post-Training Leaves Behavioral Shadows on Unrelated Decisions shows that post-training a student on just 5,664 single-word teacher answers yields +5.34pp on HumanEval+ — without the student ever seeing code.
- The key finding: post-training leaves "behavioral shadows" — stylistic and decision patterns from the teacher bleed over into seemingly unrelated domains like coding.
- Implication: distillation and post-training have hidden cross-domain spillover effects that go well beyond the target task.
More from Research
- Tempo (UIST 2026): A computer-use agent that reasons about your long-term goals — kenziyuliu · 2026-09-30
- UMAP Author Teases New Release Built for Much Larger Datasets — leland_mcinnes · 2026-09-30
- Chollet: The Test for Human-Level AGI Is Passing ARC-AGI-(n+1) on Release — fchollet · 2026-09-30
- Jev as a Reranker: a Solid Cost/Speed Default, but Not a Drop-In Silver Bullet — amaarora · 2026-09-30
- Reality Check: 4 Open-Source Models, 14,400 Real-World Rollouts to Benchmark Robot Policies — YuXiang_IRVL · 2026-09-30
- Agent kept misdiagnosing complaints — team shares 4 memory design decisions that fixed it — Financial-Shirt2304 · 2026-09-30