Omni-Woodpecker: Training-Free Hallucination Fix for Omni LLMs from Nanjing University
nanjinguniv · hf · 2026-10-26
Researchers at Nanjing University and CAS released Omni-Woodpecker (arXiv:2610.27103, MIT license), a training-free framework that reduces cross-modal hallucinations in omni LLMs.
- Problem: when evidence in video/audio conflicts with language priors (e.g. a running dog with no barking audio), existing VCD-style decoding or DPO-style alignment fails to resolve fusion-level conflicts, and alignment sacrifices general capability.
- Method: ① query-conditioned conflict diagnosis via four masking probes on the frozen model; ② evidence-preserving orthogonal decomposition of grounding evidence vs competing priors in hidden states with token-head localization; ③ conflict-adaptive representation rebalancing—localized intervention on attention heads and residual states at selected layers to suppress misleading priors and restore ignored evidence, changing no parameters.
- Backbones: Qwen2.5-Omni-7B and VideoLLaMA2-AV; code open-sourced on GitHub.
More from Multimodal
- DirCast: one streaming video model directs and commentates live Dota 2 broadcasts — Zixian Gao · 2026-10-26
- CycleGRPO unifies region understanding and grounding via self-verified RL — NationalUniversityofSingapore · 2026-10-26
- Redditor with $4,000-$5,000 budget seeks local AI video generation hardware advice — dewsphere · 2026-10-26
- Clown Prophecy: render the ending first to fix quality degradation in chained MiniMax H3 video clips — roychodraws · 2026-10-26
- Making a playable web 3D game with text: PINOC turns one sketch into rigged characters — lxfater · 2026-10-26
- RenderRepo: An apt-Style Package Manager for ComfyUI Workflows — Dr_Gimp · 2026-10-26