MAEP: training-free modality-aware pruning cuts MoE-MLLM expert memory over 90%
nota-ai · hf · 2026-11-11
Nota AI published MAEP (Modality-Aware Expert Pruning), a training-free framework for compressing MoE-based multimodal LLMs. The paper notes expert parameters dominate model size in modern MoE-MLLMs—over 90% (e.g., 54GB of 58GB in BF16 for InternVL3.5-30B-A3B and Qwen3-VL-30B-A3B)—while existing pruning methods aggregate expert importance uniformly across tokens, ignoring that the same expert may serve different roles for vision versus language. MAEP decomposes expert importance by modality in a single forward pass, using a redistribution-aware Delta-H Output (ΔH) metric to measure each expert's contribution, cutting memory while keeping competitive multimodal and text performance.
More from Multimodal
- A stunning AI-generated painting of a girl under a swirling starry sky — ciguleva · 2026-11-11
- Solo dev shares 20k-word workflow: Opus 5.5 + Seedance 2.5 for 4 viral product videos — dotey · 2026-11-11
- Vibe-coded ComfyUI node uses a decision model to auto-pick which LoRAs to load from your prompt — lurenjia_3x · 2026-11-11
- Polimon: a Pokemon-style 'founder goes to DC' game made with Fable 5.2 — zck · 2026-11-11
- Fable 5.2 one-shots a discrete component-modeled analog synth with 32x oversampling — Kyrannio · 2026-11-11
- Dev generates entire AI movie with Fable 5.2, admits he never watched it — TuhinChakr · 2026-11-11