Lancet review of 201 studies finds medical LLM bias research heavily skewed toward English and closed models

MaziyarPanahi · x · 2026-11-21

An overlooked Lancet Digital Health systematic review analyzed 201 studies on bias in medical LLMs and found heavy skew: 180 tested only closed models, 69 tested open-weight ones (24 medical-specific), just 44 used real-world data, and only 7 studied clinical note writing.

Language coverage is even more lopsided: English was tested in 32 studies, Chinese 15, Spanish 14, while Somali, Bengali, Vietnamese and Ukrainian got one study each. The authors call for large studies inside real clinical workflows with bias checks across closed and open-weight models. The poster argues open weights aren't just cheaper — they're how you actually audit bias.

Related event: Lancet Review: 90% of Medical LLM Bias Studies Test Only Closed Models(3 posts)→

Original post →

More from Safety

Safety channel →