HelixWorld 1.0: First real-time interactive audio-video world model
量子位 · wechat · 2026-08-17
NoizAI, in collaboration with researchers from HKUST, Tsinghua, CMU, and Google DeepMind, has released HelixWorld 1.0, the first real-time interactive audio-video world model. It generates continuous visuals and 48kHz binaural audio at 24FPS based on user input and camera movements. Unlike traditional dubbing, it uses a unified Transformer to generate audio and video simultaneously, ensuring spatial consistency. The team utilized a million-scale aligned dataset, action-conditioned joint generation, and causal inference with KVCache for real-time performance. Weights and code will be fully open-sourced.
More from Multimodal
- Train Your Own LTX IC-LoRA With Grok Doing the Heavy Lifting — PurzBeats · 2026-10-16
- Hugging Face launches Open TTS Leaderboard: objective metrics to rank 8K+ open TTS models — dl_weekly · 2026-10-16
- Flux 3 Klein excels at lighting and atmosphere but fumbles living subjects — Schattenmensch · 2026-10-16
- Circus clown nails tightrope stunts in impressive Seedance 2.5 video demo — azed_ai · 2026-10-16
- ComfyUI puts up $25,000 to counter Higgsfield's broken open-source promise — PurzBeats · 2026-10-16
- Underwater brand films built from one jellyfish with an open-source Minimax template — w1kke · 2026-10-16