CIViC: a four-gate protocol shows audio-conditioned visual predictors may not truly use audio
NRC-CNRC · hf · 2026-10-14
Researchers from the National Research Council Canada and CUHK introduce CIViC (Conditional Input and Visual Interpretation Checks), a four-gate protocol testing whether audio-conditioned visual predictors actually use audio as intended — since removing audio can shift outputs without proving the task uses the intended audio information.
- Links each interpretation to an intervention, held-out task comparison, or endpoint control
- Demonstrated on actor-disjoint face-raster tests, a synthetic audio-feature mixture, and source-video-disjoint SpeakerVid proxy tasks
- Notable findings: constant-audio fits can outscore audio-trained fits at a temporal endpoint; zeroing five low-level audio summaries yields lower whole-frame grayscale RMSE than aligned audio, while aligned audio only wins on segment-drift error
The takeaway: output activation, held-out error, and endpoint controls support different interpretations, arguing for stricter evidence standards in audio-visual prediction research.
More from Research
- EMNLP paper: diffusion LMs internally encode latent denoising timestep representations — gsarti_ · 2026-10-23
- KU Leuven and VIB open two Group Leader professorships in AI for Biology, start 2027 — jpillowtime · 2026-10-23
- Subspace Tuning curbs LLM catastrophic forgetting by freezing principal singular directions — scottniekum · 2026-10-23
- Perceptic launches PercepticOS, an AI workspace for 'scientific superintelligence' — nathanbenaich · 2026-10-23
- EPFL's Bosselut to talk long context as test-time learning at ELLIS seminar — andre_t_martins · 2026-10-23
- DRATS: Adaptive Task Sampling Boosts Worst-Task Performance in Multi-Task RL — Cohere · 2026-10-23