Adaptive Reward Routing improves joint audio-video diffusion via forward-process RL
_akhaliq · x · 2026-10-03
This paper proposes Adaptive Reward Routing for dynamically adapting update locations and reward coordination during forward-process RL (DiffusionNFT) of joint audio-video diffusion models. Two components: (1) Cross-Modal Influence-Guided Routing uses bidirectional cross-attention responses as a proxy for evolving cross-modal influence, dynamically reweighting token-aware losses and scaling gradients across cross-modal layers without extra model interventions; (2) Preference-Preserving Modality-Aware Reweighting keeps predefined weights as preference priors and applies branch-specific reward-gradient interactions as residual corrections after warm-up, preventing dominant rewards from suppressing weak-but-essential objectives. Experiments show consistent gains in modality quality, semantic consistency, and audio-video sync over strong RL baselines, validated by ablations and mechanism analyses.
Related event: Tencent Proposes Adaptive Reward Routing for Joint Audio-Video Diffusion RL(2 posts)→
More from Multimodal
- Core Memory profiles Ari Kuschnir, the creator behind viral AI parody videos — kyliebytes · 2026-11-19
- AI-generated flat-illustration portraits show off stylized consistency — OVolosin82152 · 2026-11-19
- GPT-6.2 Rigs and Animates a 3D Game Character End-to-End, Dev Puts It Straight Into Production — OpenAIDevs · 2026-11-19
- AssemblyAI Launches STT-3.7 Models With 86ms Agent Latency and Rewritten Inference Engine — AssemblyAI · 2026-11-19
- Pika Enact lets you direct actors and camera inside a running AI video in real time — thetripathi58 · 2026-11-19
- Google DeepMind's Genie 4 adds native sound, chainable worlds, and a community remix gallery — The Decoder · 2026-11-19