Why native multimodal models won't win it all: they can't compress visual context
gabriberton · x · 2026-10-19
Closing the thread (8/8), gabriberton pushes back on Meta's paper predicting encoder-free Native Multimodal Models (NMMs) — pixels entering the LLM through a single linear layer or small MLP, as in Gemma 4 12B, Inkling and Tuna-2. His take: NMMs are bitter-lesson-pilled and will serve many use cases, but since they can't compress the visual data fed to the LLM, they can't handle long contexts efficiently — and he expects more encoders, not fewer, in the future.
Related event: Four Years of MLLM Visual Encoding: From Cropping to Native Multimodal(8 posts)→
More from Research
- Biomedical researcher hails GPT-5 science acceleration paper with OpenAI as his most consequential work — DeryaTR_ · 2026-10-19
- Frank Nielsen paper introduces scale-of-means framework unifying Fréchet and Karcher means — FrnkNlsn · 2026-10-19
- Dust hyper-optimization delivers ~10x speedup, verified; beyond-backprop algorithms coming — burny_tech · 2026-10-19
- Leading mathematicians warn: rushing to formalize all of math with AI could do lasting damage — AlexKontorovich · 2026-10-19
- Reviewer slams 3 ICLR + 5 AAMAS submissions as too weak even for workshops — Fantastic-Nerve-4056 · 2026-10-19
- Open-source ROS 2 LiDAR localization runs 390 m outdoors on handheld Livox MID-360 — rsasaki0109 · 2026-10-19