AMBIT: Unified FOA Spatial Audio Generation and Editing from HKUST and Noiz AI
NoizAI · hf · 2026-10-27
Researchers from HKUST, Noiz AI, and UW present AMBIT, a unified framework for native First-Order Ambisonics (FOA) generation and editing.
Key contributions:
- A shared Transfusion backbone couples autoregressive scene planning with rectified-flow rendering via executable SCENEPlans specifying content, activity, and 3D trajectories; editing plans describe the full target scene, with contrastive features grounding the requested change and reference latents preserving residual acoustic detail.
- An FOA-aware continuous VAE uses shared-decoder omnidirectional recovery, asymmetric grouped KL, and spatial covariance matching to preserve content and inter-channel structure.
- On-Policy Self-Distillation (OPSD) refines planning on student-generated prefixes, with target-conditioned teachers supervising both modes.
Evaluations across speech, music, and sound show improved generation content and spatial quality, editing fidelity, and unedited-source preservation over baselines.
More from Multimodal
- Charakuru: open-source pipeline turns one illustration into a rigged VRM avatar, 1k GitHub stars — aftahi_ai · 2026-10-27
- Creator shares full breakdown and prompts for ultra-realistic AI videos — aftahi_ai · 2026-10-27
- Ornate silver Pioneer CDJ-3000X concept art made with Krea — TitusTeatus · 2026-10-27
- TIMEFLIGHT: AI-generated aerial tour through world history — ScriptLurker · 2026-10-27
- Redditor shows off Ultimate Digital Painting Moss LoRA with striking Cthulhu wallpaper art — kingroka · 2026-10-27
- Midjourney v8.2 style test: one --sref code nails moody manga illustration — azed_ai · 2026-10-27