Ling 3.0-flash-VL lands in llama.cpp: 124B MoE with only 5.5B active params

jacek2023 · reddit · 2026-09-24

inclusionAI's Ling-3.0-flash-VL has a merged PR in llama.cpp (#29151), inheriting Ling-3.0-flash's language, reasoning, and long-context abilities while adding native image and video understanding.
Key specs: 124B total parameters with only 5.5B activated per token, and context windows up to 256K tokens.
Architecture highlights:
A ViT visual encoder with a two-layer MLP projector aligning visual and text features;
VideoRoPE encodes spatial position and temporal order, enabling event localization, long-video QA, and clip editing;
A 42-layer hybrid backbone alternating KDA and Gated MLA layers at 5:1 for efficient long-context processing across text, images, video, and long agent histories;
Sparse MoE balances strong multimodal capability with inference efficiency.

More from Infra