Paper: preference models favor fluff and flattery in over 60% of cases
KordingLab · x · 2026-10-19
Kording Lab shares the paper "Flattery, Fluff, and Fog" (with a joke: old LLM says 2+2=5, new LLM philosophizes about it). Using controlled counterfactual pairs across five idiosyncratic features (length, structure, jargon, sycophancy, vagueness), the authors find preference models favor artificially biased responses in >60% of instances, are miscalibrated 40% vs human preferences, and bias features correlate weakly negatively with humans (r=-0.12) but positively with a strong reward model (r=+0.36) — evidence of overreliance on spurious cues, with mitigation strategies proposed.
Related event: Professor Mocks LLMs for Evolving From Wrong Answers to Bullshit(3 posts)→
More from Fun
- Claude and Codex blame each other for bugs, then 'fix' them — only 72% actually fixed — jkubicki · 2026-10-19
- Cloudflare's AI model ad bus and 'agentic Internet' billboard spotted in SF — threepointone · 2026-10-19
- Alexandr Wang shares his team's 'weekly status report' screenshot — alexandr_wang · 2026-10-19
- AI creator Caleb Friesen hit by conspiracy theories; peers push back — cneuralnetwork · 2026-10-19
- Industry Veteran Pushes Back: Labs Did This Before LLMs, and Researchers Barely Code — yacineMTB · 2026-10-19
- Interp researchers' parody song: what we'd do if interpretability were declared dead — ChrisGPotts · 2026-10-19