EMNLP paper: LM rewrites inflate certainty in up to 75% of outputs, 1.5-2x bias toward stronger claims
sameer_ · x · 2026-09-20
An EMNLP 2026 Main paper, "From 'May' to 'Is': Certainty Distortion in Language Model Rewriting" (arXiv:2606.07951), studies whether LMs faithfully preserve expressed certainty during meaning-preserving rewrites.
- Proposes an LM-based certainty metric aligned with population-level judgments, evaluated across model sizes and families on scientific and medical text tasks.
- Certainty distortion affects up to 75% of LM outputs, and is systematically asymmetric: most models are 1.5–2x more likely to increase certainty than decrease it.
- Effects compound over repeated paraphrasing: claude-haiku-4.5 increased certainty in 20% of medical examples after one iteration, rising to 40% after five.
- Prompt-based interventions reduce but do not eliminate the distortion — a direct warning for medical and scientific communication.
More from Research
- minpts-check: one-run predictor tells if your HDBSCAN clusters survive min_samples changes — leland_mcinnes · 2026-11-12
- Architecture study sparks monitoring concerns over a multi-hop planning spike — ChowdhuryNeil · 2026-11-12
- SGLang Megakernel Pushes Muse Spark to 1,109 tok/s, 2.1x Over Baseline — hsu_byron · 2026-11-12
- ArcGrav: a cross-model Agent Control Language to cut token overhead — ArchitecturalGravity · 2026-11-12
- Open-source dual-arm robot solves tangram puzzles with 90 demos, ~60% success — DanGrover · 2026-11-12
- FutureHouse's deslopper trains Kosmos to write indistinguishably from human experts — proteinrosh · 2026-11-12