Paper: expert disagreement labels make LLMs flip answers more but not more accurately

Yupeng Tang · hf · 2026-11-18

A controlled two-pass audit varying only the revision context shows that LLMs often fail to selectively correct errors under disagreement. Adding an "expert" label increases reversals but does not reliably improve accuracy. The authors argue revision quality should be assessed with paired conditional metrics per model and protocol rather than aggregate accuracy.

Original post →

More from Research

Research channel →