Lancet review of 201 studies finds medical LLM bias auditing is rare, argues open weights are key
MaziyarPanahi · x · 2026-11-21
An open-access scoping review in Lancet Digital Health (Liao, Li, Yang et al., Nov 18) systematically covers 201 studies on bias and equity in medical LLM applications.
Key numbers:
- 180 studies tested closed models; 69 tested open models, 24 of them medical-specific
- Only 44 used real-world data
- Just 7 examined clinical note writing
The poster argues open-weight models aren't just cheaper — they're the only realistic way to independently audit bias in healthcare LLMs.
Related event: Lancet Review: 90% of Medical LLM Bias Studies Test Only Closed Models(3 posts)→
More from Safety
- Christiano: Two Months of Math Progress in a Decade, and ML Could Be Next — OwainEvans_UK · 2026-11-21
- AI safety researcher: doubting China's moratorium compliance is like doubting your own marriage — DavidSKrueger · 2026-11-21
- Hinton may be softening on an AI moratorium after SF risk-scenarios panel — DavidSKrueger · 2026-11-21
- 205 Protesters Rally Outside Anthropic Against Recursive Self-Improvement — DavidSKrueger · 2026-11-21
- Study: human auditors can't detect sabotage in AI safety research — ronbodkin · 2026-11-21
- ~$50K CAD unauthorized Gemini API usage on a partner-billed account sparks routing nightmare — Big-Coffee-8899 · 2026-11-21