Study of 18 VLMs finds answer inertia: CoT reasoning rarely revises initial predictions
delliott · x · 2026-10-06
A University of Copenhagen team (Danae Sánchez Villegas, Desmond Elliott et al.) presents at COLM an analysis of reasoning dynamics across 18 vision-language models, spanning instruction-tuned and reasoning-trained models from two families.
Key findings:
- Answer inertia: models reinforce early commitments during Chain-of-Thought rather than revising them; open-weight VLMs largely do not revise predictions during CoT.
- Reasoning-trained models show stronger corrective behavior, but gains depend on modality conditions from text-dominant to vision-only.
- Controlled interventions with misleading textual cues show models are consistently influenced even when visual evidence suffices; their longer, fluent CoTs can appear visually grounded while actually following textual cues.
- Detectability of this influence varies across models and monitoring targets, bounding CoT-based monitoring of modality reliance.
More from Research
- Proactivity-Gym: 23-agent study shows correct work alone can't sustain user trust — minnesotanlp · 2026-10-06
- SearchJev makes search-agent decisions 5x faster with 41-74% lower calibration error — Congfeng Cao · 2026-10-06
- LLM autoresearch builds AutoSella, cutting DFT force calls to 40-77% of Sella's — Artem Tsypin · 2026-10-06
- ID-Forcing extends short-horizon video diffusion models to minute-scale generation — SeoulNatlUniv · 2026-10-06
- Self-generated feedback destabilizes test-time training, causally decomposed at 128K tokens — KAUST · 2026-10-06
- PFLM: a 300M model pretrained on zero real languages learns languages in context — Lennart Carstens-Behrens · 2026-10-06