Adobe's OmniXRef unifies text, image, and video references for video editing
adobe-research · hf · 2026-10-20
Adobe Research and Brown University introduce OmniXRef, a unified framework for multimodal multi-reference video editing that lets users combine any number of text instructions, image references, and video references to control attributes of a source video. Key points: - A reference-bank input design plus R²-RoPE for generalizing to more references. - OmniXRef-Model uses a single-stream diffusion transformer to jointly process the source video and MLLM-provided multimodal context in an interleaved sequence. - To overcome scarce video-reference training data, they build OmniXRef-Data via synthesis pipelines with curriculum learning. - OmniXRef-Bench evaluates video-reference attribute transfer across Camera, Motion, Content, Style, and VFX. The model outperforms task-specific baselines and commercial models.
More from Multimodal
- AI-Generated Sci-Fi Short 'Alien Invasion Prologue' Wows Reddit With Cinematic Look — theodore_70 · 2026-10-20
- AI-Generated Harry Potter Clip Shows Harry and Hermione's Confession Scene — lightandcolorlab3 · 2026-10-20
- HeyGen Video 1.0 Launches, Ranks #3 on Design Arena and Makes HeyGen #2 Video Lab — HeyGen · 2026-10-20
- Prototyping with Gemini 4 Argon: WebGL toys, auto-edited reels, procedural C++ music — alexanderchen · 2026-10-20
- TwelveLabs Pegasus 1.5 lands on Amazon Bedrock: turn 2-hour videos into timestamped JSON — AWS ML Blog · 2026-10-20
- Sunra MCP aggregates top image/video/audio models into Claude and ChatGPT at maker API prices — dr_cintas · 2026-10-20