Lluna: Collaborating Small Unimodal Models Beat Missing-Modality Baselines by 4.26%
jindong_wang92 · x · 2026-10-16
Researchers from William & Mary, CMU, Microsoft, and UChicago introduce Lluna, a framework that lets existing small unimodal models on Hugging Face collaborate via a graph to handle missing-modality inference—no large multimodal model or multi-agent setup required.
- Problem identified: collaboration gains weaken or reverse to negative when modalities go missing, due to poor recovery of modality-specific signals and state-dependent representations
- Method: a direct private prediction path from adapted unimodal features plus a graph-based collaboration branch for residual correction, organized by Progressive Collaboration Consistency from richer to poorer inputs
- Results: across 8 datasets spanning affective computing, scene understanding, and medical prediction, Lluna improves Mean-Missing by 4.26% on average over the strongest baseline, keeping positive collaboration gains on all tasks under minimal inputs
Paper and code are public.
Related event: Lluna Lets Small Unimodal Models Collaborate to Beat Large Omni-Modal Ones(3 posts)→
More from Multimodal
- Saturday Challenge #129 launches with 'Reality Glitch' theme for AI image prompts — LudovicCreator · 2026-10-17
- FluidNinja LIVE-2 Ships After 3 Years: Real-Time Fluid Sims in UE5 — Promptmethus · 2026-10-17
- CapCut × Codex turns ChatGPT into a conversational AI video editor — anthara_ai · 2026-10-17
- MOSS-Image samples impress Redditors as Fudan team's new image model surfaces — Only-Baseball-3129 · 2026-10-17
- ComfyUI MOSS-GGUF already supports FP16 — a one-line fix speeds up older Turing GPUs — fjgcudzwspaper-6312 · 2026-10-17
- AI-generated flying Cybertruck meme: statue of David at the wheel over Rio — econoar · 2026-10-17