FoleyForge: instruction-based sound effect editing model hits SOTA instruction adherence
kastnerkyle · x · 2026-11-09
FoleyForge (arXiv:2611.08695) is a generative model for instruction-based editing of sound effects: given a clip and a free-form instruction, it applies the edit while preserving the input's identity. One model handles semantic, stylistic, and DSP-like changes. The authors introduce a data pipeline that uses an audio LM to author DSP-based acoustic edits and similarity-preserving semantic edits, addressing the lack of public audio editing data, then fine-tune a rectified-flow audio model with attribute controls (loudness, edit strength, input-derived contour envelope) tunable via classifier-free guidance. It achieves state-of-the-art instruction adherence at competitive audio quality and source faithfulness.
More from Multimodal
- Kling 4.0 prompt demo shows how to generate dynamic action scenes with camera movement — umesh_ai · 2026-11-09
- Video-Coder: first code-driven video generation agent trained with execution rewards — UCF-Ai-LLMs · 2026-11-09
- OmniLatent: unified latent for part-aware 3D asset generation from a single image — Yunhan Yang · 2026-11-09
- Midjourney sref 4015763981 Renders Cartoon IPs as Glossy Art Toys — azed_ai · 2026-11-09
- GPT image 2 fashion editorial poster: full structured prompt and negative prompt shared — SimplyAnnisa · 2026-11-09
- Mistral used Voxtral 3 Transcribe to live-caption every AI Engineer Paris talk, with context biasing — MistralAI · 2026-11-09