OmniDelta trims OmniLLM audio-video tokens with skill-driven budget allocation
Haoyang Huang · hf · 2026-07-29
OmniDelta reallocates token budgets for audio-video compression in OmniLLMs
OmniDelta is a training-free, skill-driven framework for token compression in omni-modal LLMs. The paper argues that existing pruning methods focus too much on selecting tokens under a fixed budget, while the harder problem is how to allocate that budget across modalities and within each modality.
What it does
- Builds audio and video skill pools to shift budget based on query intent.
- Reallocates budgets across audio segments and video frames using local complexity and temporal redundancy.
- Can be combined with existing pruning methods while keeping the same total retention ratio.
Results
- Evaluated on four audio-video benchmarks with two Qwen2.5-Omni models.
- The method establishes a new accuracy-efficiency Pareto frontier across pruning ratios.
- At 25% token retention on Qwen2.5-Omni-7B, it cuts GPU memory by 22.0% and delivers a 1.64× end-to-end speedup over full-token inference.
More from Multimodal
- Describe your dream world to an AI dragon, which generates the planet for you — repligate · 2026-08-24
- Using kintsugi texture to fix cracks in edited 3D meshes — repligate · 2026-08-24
- Generating Hannibal Character Videos with FL2VA Model — Nimblecloud13 · 2026-08-24
- MiniMax H3 Revives Medieval Short Stories: Complete Workflow Shared — zanatas · 2026-08-24
- NAPE Audio Pretraining Achieves SOTA Without Decoders — kastnerkyle · 2026-08-24
- H3 excels at generating complex space scenes — SIR_NVAX_A_LOT · 2026-08-24