AI2's FairStress benchmark exposes hidden fairness asymmetries across 11 open-weight models
allen_ai · x · 2026-10-21
Allen AI continues its FairStress thread with test results: the team tried 9 approaches for reducing fairness gaps, and most didn't reliably improve symmetric robustness — some even hurt performance.
- Older benchmarks like BBQ can miss overcorrection, but FairStress reveals the hidden asymmetries they overlook.
- Across 11 open-weight models, explanations for minority-coded choices leaned on more positive, identity-related language, while majority-coded choices drew on terms like "evidence" and "criteria."
More from Models
- Space Bunny Alpha revealed as MiniMax M3.1 Flash: 1M context, budget-friendly coding — ZabihullahAtal · 2026-10-21
- 'Opus 5.5 Medium' Spotted in Rork's Model Picker, Hinting at New Anthropic Model — rudrank · 2026-10-21
- RSI-Bench Clears First Task Set Through Reward-Hacking Detection Pipeline — SRSchmidgall · 2026-10-21
- Dev complains Anthropic's bio safeguards trigger just from reading a bio paper — robleclerc · 2026-10-21
- Redditor claims uncensored local DeepSeek build answers virus-writing prompt with zero hesitation — Only_Light_9817 · 2026-10-21
- Interconnects Artifacts 25: 28 new models, DeepSeek V4.1 Flash, and US-China regulatory controls — natolambert · 2026-10-21