Mirage: A 22MB VLM Runs Detection, OCR and Segmentation in One Pass at 74ms on Phone NPU

Henrie_the_dreamer · reddit · 2026-11-07

Cactus Compute released Mirage, a 22MB (51.3M-param) vision-language model for phones, wearables and robots that does scene description, object detection, OCR, segmentation, spatial relations and decisions in a single pass, in 7 languages.
Measured end-to-end on a Samsung Galaxy S26: 74ms on NPU, 107ms on CPU. Architecture: 7.8M encoder (HConvGLU with fixed Hadamard transforms), 40.8M decoder (Simple Attention + Hadamard MLP), 2.6M OCR/spatial heads; detection and masks share image features.
Thesis: small models shouldn't spend capacity on prose — put structure in the problem via spatial heads and short references linking names to regions.
Key finding: feeding Mirage's structured record to a text-only LLM may grant VLM ability without fine-tuning. Qwen3-0.6B jumped from 9.3 to 31.3 on 256 VQAv2 questions; Qwen3-4B from 33.4 to 37.3. But it's not universal — Nemotron 3 Nano 4B collapsed from 33.1 to 5.8 due to JSON format failures, and perception misses (a missed skateboard) can mislead the larger reader. Fine-tuning the reader is next.

More from Embodied