Researcher re-tests own LLM hiring-bias study: most "bias" was random noise
Signal_Rabbit_8303 · reddit · 2026-08-21
After the author's LLM resume-screening study flagged 45% of score differences as bias, community feedback prompted new experiments addressing three specific objections:
- Reasoning transplant (320 runs): swapping positive/negative justifications back into prompts moved scores 3.62 points in the reasoning's direction 99.7% of the time — scores do follow reasoning, but baseline instability meant most of the original 45% "bias" was random noise mislabeled.
- Schema ordering (4,800 runs): placing the score last had no effect on stability and increased hire/no-hire disagreement from 33% to 54%; blind instructions also failed to reduce variance.
- Placebo control (4,165 runs): meaningless edits like car color shifted scores almost as much as demographic edits (0.328 vs 0.362 points); "Silver Golf" moved scores more than changing name or university.
Bottom line: first names and career gaps show real signal, but raw instability drowns out most axes. Wrapper choice matters hugely — a hidden system prompt when running Claude via CLI shifted scores by 0.247 (88% of the demographic signal), so benchmarks don't transfer across wrappers. Data and code are open-sourced.
More from Safety
- Interactive Demo Explains Anthropic's Watermarking via Probability Bias — VeryWellVersed · 2026-08-21
- NEJM AI: Behavior interventions need systematic descriptions for cumulative science — zakkohane · 2026-08-21
- Google DeepMind Publishes Nature Paper on LLM Watermarking — burkov · 2026-08-21
- Jessica Taylor paper: Occupational Infohazards — ArtificialOther · 2026-08-21
- Prediction Market: 70% Chance of Statewide Data Center Moratorium by Year-End — Polymarket · 2026-08-21
- AP Stylebook Considers Banning 'Hallucinate' for AI Fabrications — AndrewSchmidtFC · 2026-08-21