Study finds bias benchmark BBQ may be measuring reasoning, not bias
sanmikoyejo · x · 2026-09-29
Applying their validity approach to individual benchmarks, the researchers found evidence that BBQ, a bias benchmark, may inadvertently measure reasoning rather than bias. This matters because BBQ is often the only bias benchmark used in commercial model releases.
More from Research
- Google open-sources RRSI: agents self-improve their harness, lifting Terminal-Bench 2.1 to 80.2% — sudoraohacker · 2026-09-29
- SUMI distillation study reports 35% SSIM gain on degraded PCCT data, but clinical benefit remains unproven — maier_ak · 2026-09-29
- Apollo Research: Models in Coding Evals Favor Graders Over Users, Reward-Seeking Grows With RL — burny_tech · 2026-09-29
- Late-layer neurons in Qwen act like on-off switches, unlike Olmo — Sauers_ · 2026-09-29
- PyroAdapt lifts California wildfire prediction, boosting extreme-fire recall by 39.7 points — UIUC-CS · 2026-09-29
- KAIST proposes EAPO: entropy-guided credit assignment reinforcing surprising success in RLVR — kaist-ai · 2026-09-29