CoVeR: Drop 92% of Visual Tokens, Keep 93.5% Performance in Multi-View 3D VLM Reasoning
Scobleizer · x · 2026-09-10
CMU Robotics and Meta propose CoVeR, a deterministic, training-free visual token pruner for multi-view 3D reasoning in 2D VLMs.
The problem: Feeding multiple views to a 2D VLM explodes token counts — LLaVA-OneVision-7B yields 8,748 visual tokens at 12 views, and 31% are spatial duplicates of the same surface seen from another camera. Existing attention/feature-based pruners handle semantic redundancy poorly, keeping near-duplicates of prominent regions while leaving the rest of the scene unrepresented.
Method: CoVeR uses 3D coordinates alone — no attention, no visual features, no retraining. Posed images and depth are back-projected into 3D feature points; coverage initialization removes cross-view duplicates via an adaptively-sized voxel grid, and coverage expansion fills underrepresented regions until the exact token budget is met.
Results:
- At 8% of visual tokens, retains 93.5% of full-token performance (avg. of 3 benchmarks);
- +3.9 pts average accuracy over the prior SOTA pruner at aggressive 8% budgets;
- At 14% retention: 8.6× fewer LLM TFLOPs and 7× smaller KV cache, with only 1.1% relative drop on ScanQA.
An interactive demo lets you hover any camera to see exactly which tokens are kept; code is coming soon. Robert Scoble notes this "use less data, get similar results" property matters for robotics.
More from Embodied
- π0.5 study: raising demos from 10 to 438 lifts Place Ball in Bowl success from 7% to 83% — stingning · 2026-10-27
- GPT-6 Astra controls dexterous hands zero-shot at just 5.2% success in 500 trials — stingning · 2026-10-27
- HandsOn benchmark: 66 verified tasks and 18,000 human demos for dexterous manipulation — stingning · 2026-10-27
- HandsOn benchmark: 66 human-verified tasks and 18,000 curated demos for dexterous manipulation — stingning · 2026-10-27
- HandsOn splits evaluation: Core-50 execution vs Transfer-16 zero-demo transfer — stingning · 2026-10-27
- Best policy hits only 29.1% on HandsOn; tool use drops to 2.9%, transfer near zero — stingning · 2026-10-27