KAIST's VETO suppresses text-only shortcuts hidden behind VideoLLM SFT gains

Youngwoo Shin · hf · 2026-11-20

A KAIST/PYLER paper shows VideoLLMs learn a shortcut during SFT — answering from textual regularities while bypassing the video — which inflates in-distribution accuracy without real gains. VETO (Video-Evidence Training via Orthogonal projection) calibrates the text-only activation subspace of the base model and projects SFT updates onto its orthogonal complement, suppressing the shortcut as it forms. Across seven VideoLLMs it preserves SFT gains and beats SFT on six out-of-shift benchmarks.

Original post →

More from Research

Research channel →