Structured captions boost text-to-music alignment, new Music DSG metric beats CLAP
affige_yang · x · 2026-11-04
An arXiv paper (Junkai Wu et al., with Nicholas Bryan and Mari Ostendorf) proposes fine-tuning a flow-based DiT music generation model on structured captions (SCAPs), with controlled ablations comparing free-form vs structured captions, partial vs full coverage, and global vs temporal text info, plus a multi-stage inference-time prompt rewriter. They also introduce Music DSG, a Davidsonian Scene Graphs-inspired evaluation protocol for fine-grained, interpretable text-alignment measurement. Results: SCAPs fine-tuning significantly improves text alignment while also improving quality, and Music DSG correlates with human preference better than CLAP. Submitted to ICASSP 2027.
More from Research
- Solo researcher builds foundation model from scratch using Lie groups and differential geometry — ReportProfessional57 · 2026-11-04
- Nanjing University's SeMo3D Unifies 3D Generation, Editing, and Completion in One Model — nanjinguniv · 2026-11-04
- RoboColosseum Benchmark Finds Weak Cross-Embodiment Agreement in Robot Policies — huggingface · 2026-11-04
- SpecPilot: Berkeley and NVIDIA's Agent Loop Auto-Optimizes Speculative Decoding per Deployment — UCBerkeley · 2026-11-04
- Adobe's MoviEdit Tackles Long-Form Multi-Shot Video Editing With 87K Auto-Generated Pairs — adobe · 2026-11-04
- Alibaba's Recursa Pushes Recursive Self-Improvement, Lifting Firm Benchmark Pass Rate to 55% — alibabagroup · 2026-11-04