Evaluating Music Context Preservation: A Multi-facet Framework for Music Editing Systems
Yash Vishe, Eric Xue, Xunyi Jiang, Zachary Novack, Junda Wu, Julian McAuley, Xin Xu
cs.SD, cs.AI
2025-12-17
MuseCPEval scores whether music editors keep untargeted facets. Ten metrics cover harmony, rhythm, form, and melody; human agreement is 100% on rhythm, 45.5% on melody.
Text-guided music editors can now swap timbre, substitute instruments, and change genre. Evaluation still mostly asks whether the result sounds good. It rarely asks, systematically, whether the attributes that should have stayed put actually stayed put. The paper calls that Music Context Preservation, MuseCP.
When prior systems do measure preservation, they pick one or two facets. ZETA checks structure, Audio Prompt Adapter checks harmony, SteerMusic checks melody. AUDIT and Instruct-MusicGen, in this paper's census, skip MuseCP entirely. Without a shared toolkit, a "successful" style transfer can be a timbre change that also rewrote the meter and the form.
Given source audio x and edited audio x', MuseCPEval scores four facets with ten metrics.
Harmony: circle-of-fifths distance CoF after mapping minor keys to their relative majors (0 same key, 1 maximally far); global chroma cosine; chroma DTW similarity for time-varying chord content.
Rhythm and meter: folded BPM difference, collapsing octave errors such as double or half tempo; Beat F-measure with a ±70 ms hit window; Information Gain on beat-phase error, high when errors concentrate, low when they spread uniformly.
Structure: msaf segments scored with pairwise F-measure and Adjusted Rand Index, rewarding both "together" and "apart" agreement.
Melody and motif: contour DTW; interval 3-gram recall, the fraction of short source motifs that survive.
An appendix, added after reviewer comments, scores timbre via symmetrized KL on MFCC Gaussians and mean-MFCC cosine. The main text stays with the four facets above.
Validation has two tracks. Objectively, 50 Lakh MIDI clips with clear melody, meter, and form receive eight single-facet edits, then FluidSynth rendering; metrics should move only on the targeted facet. Perceptually, each facet has four pairwise comparisons at two edit strengths. Listeners pick which clip is farther from the source on that facet. 33 questionnaires came back, 11 passed sanity checks.
Case studies follow each system's official protocol, so scores are not comparable across systems: MusicMagus on 60 style or instrument pairs; ZETA on 324 MedleyDB text edits; Audio Prompt Adapter on 40 clips times 3 instrument instructions; Instruct-MusicGen on 150 Slakh pairs per add, remove, or extract stem task.
Under controlled edits the metrics mostly move as designed. A +7 semitone shift yields mean CoF 0.18 against a theoretical 0.167, with rhythm metrics nearly still. Tempo +50% yields folded BPM difference 26.10, BeatF 0.24, IG 0.19. A uniform +150 ms beat shift collapses BeatF to 0.03 (outside the 70 ms window) while IG stays at 0.63, because a fixed phase offset is not uniform noise. Structural scores never hit 1.0 even when form is untouched, since the segmenter reacts to spectral change, but they still separate form-changing edits from form-preserving ones.
Perceptual alignment is uneven. Rhythm: metric, gold, and human agree 100%. Harmony: metric-gold 93.2%, metric-human 65.9%. Melody versus human is 45.5%. Melody is the least stable facet here.
The four editors are diagnostic portraits, not a ranking. Audio Prompt Adapter holds ChromaSim 0.95 and StructPairF 0.88, so high-level harmony and form survive, while ΔBPM hits 20.856. Instruct-MusicGen's ΔBPM is 20.012, with weak rhythm and melody, consistent with training only light fusion and not beat alignment; adding or removing a stem can also carry the lead line away. MusicMagus ΔBPM is 5.573 and ZETA 4.253, so global tempo is steadier, yet MusicMagus BeatF and IG sit near zero, which the paper reads as style-induced rhythmic change rather than an obvious bug. ZETA's DDPM inversion pins the source latent trajectory; its harmony and beat preservation match that design.
Music-editing teams get a split scorecard for "what should not have moved," which is more useful as a regression test than a single aesthetic number. High chroma and near-zero BeatF means the model kept the key and dropped the pulse: fix time alignment, do not add another beauty score. Code and a demo are public.
This is an evaluation framework, not a new editor. The four case protocols differ, so they are not a system leaderboard.
Objective checks edit in the symbolic MIDI domain and then render. Real diffusion edits entangle spectral changes; equally clean facet isolation in the wild is unproven. Structural metrics drop even under transposition because the segmenter is spectral, so relative ranking is usable, absolute "perfect form" is not. The listening study keeps 11 valid raters, and melody-human agreement is 45.5%, so that facet is currently a reference signal. The four systems never share a dataset or instruction set; diagnoses inherit each official setup. Timbre lives in the appendix. If the edit's goal is an instrument swap, timbre preservation should not be a penalty, and the paper has no official target-facet mask protocol for turning metrics off.