OmniScope: Modality-Decoupled Token Compression for Omnimodal Large Language Models
Jinsen Su, Yongdong Luo, Yuexiao Ma, Yibo Hu, Meiguang Jin, Xiaowu Zheng
cs.CV
2026-07-25
OmniScope decouples audio and video token compression in omni-modal LLMs: each modality scores its own query relevance. At 25% retention on Qwen2.5-Omni it hits 3.53x prefill, 15% less memory, only a 0.35-point drop.
Omni-modal large language models (OmniLLMs) such as Qwen2.5-Omni and GPT-4o ingest video frames and audio at the same time, and the token count climbs with video length. A few minutes of video can flood GPU memory, and the prefill stage gets slow, which rules out real-time and edge deployment.
The obvious fix is token compression, but the hard part is which tokens to drop. Existing omnimodal methods (OmniZip, OmniSIFT) rely on unidirectional cross-modal guidance: one modality's importance score decides what the other keeps, e.g. audio telling you which video frames matter. The paper shows this assumption breaks often. For the same query, audio-critical and video-critical moments frequently land at different times. They quantify it: across 1,197 query-video pairs, about 78.3% show only weak correlation between audio and visual salience. A whistle peaks early in audio; the referee's hand signal peaks late in video. Ask what the referee signals after the whistle, and audio-guided video pruning throws away exactly that signal.
A counterintuitive observation comes first. With full tokens, attention in Qwen2.5-Omni is locked inside local time windows; cross-window and cross-modal attention is negligible. Compress redundant tokens away, and that cross-window and cross-modal attention strengthens. Compression is not just discarding information; it releases the attention budget that redundant tokens were diluting and redirects it. The real question is whether the released attention lands on task-relevant cues or on secondary content.
OmniScope's design principle fits one line: share the query across modalities, but not the salience estimates. Each modality scores its own importance against the query and allocates its own budget. The framework is training-free and plugs into Qwen2.5-Omni at 7B and 3B.
Three stages.
Four audio-video benchmarks (WorldSense, DailyOmni, OmniVideoBench, Video-MME) against Random, FastV (A&V), and OmniZip, the current strongest open-source training-free omnimodal method. OmniSIFT is excluded because it needs extra training and has no public code.
| Setting (Qwen2.5-Omni-7B) | Full Tokens | OmniScope | OmniZip | FastV |
| 45% retained, avg | 51.35 | 51.65 (+0.30) | 51.15 (-0.20) | 51.13 (-0.22) |
| 25% retained, avg | 51.35 | 51.00 (-0.35) | 49.80 (-1.55) | 50.53 (-0.82) |
At 45% retention the 7B model actually gains 0.30 points. Pushed to 25%, OmniScope drops only 0.35 while OmniZip drops 1.55. The 3B model follows the same trend. OmniScope posts the highest average among compression methods at every setting.
Efficiency (7B, WorldSense):
| Metric | Full Tokens | 25% retained |
| Prefill time | 6299 ms | 1784 ms (3.53x) |
| Peak GPU memory | 28.31 G | 24.00 G (-15.2%) |
| End-to-end latency | 10.97 s | 4.79 s (2.3x) |
Ablations verify the core claim. Replacing query-aware budgets with uniform compression across both modalities costs 1.65 points; both audio-guides-visual and visual-guides-audio, in either direction, underperform independent query-based compression. All-Anchor or All-Delta vision strategies lose 1.10 and 1.65 points respectively. Per-second audio merging beats energy-based filtering (-2.75), random drop, and average pooling.
OmniLLMs are multiplying, and the joint audio-video token length is the practical bottleneck for deploying them. OmniScope offers a training-free acceleration path that drops straight into existing inference pipelines: 25% retention buys 3.53x prefill speedup and 15% less memory at almost no accuracy cost.
The more valuable asset is the principle itself: share the query, not the salience. It is not tied to this framework; any system compressing tokens across multiple modalities can borrow it. For anyone working on long-video understanding, real-time audio-video interaction, or edge deployment, this is a directly usable engineering result.
The biggest one, admitted by the authors: vision needs an external CLIP scorer running outside the LLM inference pipeline. The consequence is that at 45% retention the end-to-end latency (5.28 s) is actually slower than FastV (4.58 s) and OmniZip (4.51 s). Shorter generation makes this fixed cost loom larger (about 13% of end-to-end latency at 1 generated token, down to 7% at 100 tokens). All evaluation tasks are short multiple-choice QA, where prefill dominates and the speedup looks great; long-output scenarios like captioning get only an amortization argument, not measured numbers.
The two scorers are asymmetric: vision needs external CLIP while audio reuses the model's own encoder. The paper notes in an appendix that the model's internal vision-text alignment is not yet reliable enough for scoring.
Two things feel under-verified. "Best at all compression settings" really means two settings (45%, 25%) across four benchmarks, which is narrow. And Video-MME is fundamentally a video benchmark that the paper itself says audio further improves, so counting it among audio-video benchmarks is a slight stretch. Validation covers only Qwen2.5-Omni at 7B and 3B, with no other OmniLLM tested.