DeepMind paper: without direct image viewing, Gemini's defect-spotting falls from 86.7% to 14.3%

nukebuddy · reddit · 2026-11-22

A Reddit post digs into DeepMind's Multimodal Knowledge Agent tech report, using a road-inspection case to show why agents must actually open images. Given direct image viewing, Gemini 4 Carbon spotted a pothole in every run; without it, the same model and setup fell back to OCR, OpenCV, even ASCII renderings of photos—and missed it every time. Scores on harder criteria dropped from 86.7% to 14.3% while the no-viewing variant used 2.2× more output tokens; full-benchmark drops were smaller (6 points) and GDPval barely moved. Practical takeaway: when auditing agent work on scans or drawings, ask it to point to the actual evidence on the page—a convincing spreadsheet can be built from an incomplete reading.

Original post →

More from coding & agent

coding & agent channel →