New VLM explanation scores improve correctness judgments by 11.1% in user study
_jessethomason_ · x · 2026-07-28
A USC and Carnegie Mellon team proposes two quality scores for VLM explanations: Visual Fidelity, which measures how faithful an explanation is to the visual context, and Contrastiveness, which measures whether the explanation highlights details that separate the prediction from plausible alternatives.
On A-OKVQA, VizWiz, and MMMU-Pro, these scores are better calibrated with correctness than prior explanation-quality measures. In a user study where participants judged whether VLM predictions were accurate without seeing the image, showing the scores improved accuracy by 11.1% and reduced the rate of falsely believing incorrect answers by 15.4%.
Related event: ACL 2026 Paper Proposes New Metrics for VLM Interpretability(2 posts)→
More from Research
- Allen AI's OlmoEarth Maps N. America Wildfire Risk with 155x Speedup — allen_ai · 2026-07-29
- Researcher Outlines Technical and Safety Challenges of Open-Weight AI — StephenLCasper · 2026-07-29
- Anthropic hires MIT researcher focused on pretraining and ecosystem safety — ShayneRedford · 2026-07-29
- New flow-map framework lets generators expand output size on the fly — gottapatchemall · 2026-07-29
- Three-finger dexterous hand debuts as a simpler robot-hand design — Darpinian · 2026-07-29
- Kimi K3 paper lands on arXiv with architecture notes — yogthos · 2026-07-29