Paper Reveals VLMs Answer Without Images, Multimodal Scores Inflated
ziv_ravid · x · 2026-07-04
In the "Mirage Probes" paper, researcher zivravid found that Vision Language Models (VLMs) can correctly answer image-based questions even without an attached image. This "hallucination mirror" effect (Asadi et al.) causes multimodal benchmark scores to be systematically inflated. The paper explores the mechanism behind this phenomenon, highlighting potential biases in current multimodal evaluation systems and offering insights for more accurately assessing true visual understanding.
More from Models
- Nanbeige4.3-33B benchmarked: 59 tok/s single stream, ~1,060 tok/s across 64 concurrent runs — MaziyarPanahi · 2026-10-26
- Only 2 of 25 full ChatGPT rewrites flagged 100% AI by detectors — paulnovosad · 2026-10-26
- Qwen 4 Reportedly Spotted in the Arena, Hints at Alibaba's Next-Gen Model Testing — AIFlow_ML · 2026-10-26
- Nathan Lambert's first open models lecture: distillation, risks and the ecosystem explained — natolambert · 2026-10-26
- Nanbeige 4.3-33B flops real-world test: zero compilable output vs Qwen — julianharris · 2026-10-26
- Claude's safety classifier blocks SS7 defense research, pushing a security dev to an abliterated GLM — l4rz · 2026-10-26