MIT-IBM Team Finds Multimodal AI Models Can Ace Benchmarks on Pure Noise

MIT News AI · rss · 2026-11-21

An MIT News Q&A with Jim Glass's MIT-IBM team covers key failures of multimodal foundation models: models like GPT-4o can't localize objects from few-shot examples (fixed via a pseudonym-substitution trick), and audio-LLMs often game multiple-choice benchmarks without listening — they still score well on pure noise. The team's approach emphasizes weakly supervised, data-efficient adaptation (including test-time RL) and public benchmarks, and is now moving to full-duplex spoken dialogue systems that listen while speaking.

Original post →

More from Multimodal

Multimodal channel →