OmniDelta trims OmniLLM audio-video tokens with skill-driven budget allocation
Haoyang Huang · hf · 2026-07-29
OmniDelta reallocates token budgets for audio-video compression in OmniLLMs
OmniDelta is a training-free, skill-driven framework for token compression in omni-modal LLMs. The paper argues that existing pruning methods focus too much on selecting tokens under a fixed budget, while the harder problem is how to allocate that budget across modalities and within each modality.
What it does
- Builds audio and video skill pools to shift budget based on query intent.
- Reallocates budgets across audio segments and video frames using local complexity and temporal redundancy.
- Can be combined with existing pruning methods while keeping the same total retention ratio.
Results
- Evaluated on four audio-video benchmarks with two Qwen2.5-Omni models.
- The method establishes a new accuracy-efficiency Pareto frontier across pruning ratios.
- At 25% token retention on Qwen2.5-Omni-7B, it cuts GPU memory by 22.0% and delivers a 1.64× end-to-end speedup over full-token inference.
More from Multimodal
- Fish Audio's open-weight Drama 3 voice model claims top emotive quality across 83 languages — mhdfaran · 2026-10-16
- Claude Opus 5.5 makes a pure-code Halloween animation starring bots trick-or-treating for tokens — sattvi · 2026-10-16
- 1,290-line commit lands MOSS Image support in ostris/ai-toolkit — ostrisai · 2026-10-16
- Ostris AI Toolkit adds training support for OpenMOSS's MOSS-Image-SFT model — ostrisai · 2026-10-16
- Nokia vs iPhone, car edition: AI image shows Tesla Roadster hovering over a Porsche — vaibhavbetter · 2026-10-16
- Midjourney multi-sref weight mixing recipe for cinematic light & motion portraits — michaelrabone · 2026-10-16