MIT's UQ-DAAAM helps robots know when their memory is wrong, boosting accuracy by 5%

MIT News AI · rss · 2026-11-05

MIT researchers introduced UQ-DAAAM (Uncertainty Quantification for Describe Anything, Anywhere, at Any Moment), a system that detects and corrects mislabeled object descriptions in a robot's memory. It builds on DAAAM, which uses vision-language models (VLMs) to describe objects into a 3D map-based memory — but robots can mislabel objects seen from far away or in blurry images (e.g., mistaking a chimney for a squirrel), and such errors persist.

How it works:

Compares multiple descriptions of the same object from different viewpoints, fusing VLM confidence with cross-view consistency into a single uncertainty score;
When uncertainty exceeds a threshold, a mathematically proven heuristic selects additional views that best reduce uncertainty — without rerunning the VLM, cutting compute dramatically;
An LLM then fuses the captions (weighting newer, less ambiguous ones) into a refined memory entry, updated in real time on a moving robot.

Results: With only two extra captions per object, it resolves ambiguity for 92% of objects and improves object-description accuracy by 5% over state-of-the-art, with better answers about object locations and sighting times.

The work, by Luca Carlone's group (co-leads Harry Zhang and Nicolas Gorlo), will be presented at NeurIPS. Next steps include mobile manipulators and actively revisiting high-uncertainty areas. Carlone notes physical AI must "know what it doesn't know" to be deployed safely.

Original post →

MIT's UQ-DAAAM helps robots know when their memory is wrong, boosting accuracy by 5%

More from Embodied