SPDMX: Large-Scale Open Stem Audio From Symbolic Music
Phillip Long, Eduardo Escoto, Julian McAuley, Zachary Novack
cs.SD, cs.AI, cs.LG, cs.MM, eess.AS
2026-11-06
UCSD renders public-domain PDMX into 253,982 CC BY 4.0 songs and 490,159 mono stems (5,469 h); 88,403 multitrack songs cover 2,734 h, above Slakh2100's 145 h.
Source separation and stem-conditioned synthesis need the tracks before the mix. A stem is one instrument or vocal part, stored alone. Finished mixes are easy to collect. Stems you can legally redistribute are not, because copyright keeps commercial multitracks out of public training sets.
Realism, scale, and licensing do not arrive as a package. Recorded sets stay small and closed: MUSDB18 is 150 songs and 10 hours, the two MedleyDB releases together are 196 songs and 13.8 hours, and MoisesDB is 240 songs and 14.4 hours. None is an openly licensed download. Slakh2100 synthesizes 2,100 songs and 145 hours that you can download as CC BY audio, but the instruments are commercial libraries you cannot pass on. CocoChorales reaches 240,000 songs and 1,411 hours on a redistributable synthesizer, and the music is generated chorales, not an existing repertoire. EnsembleSet is 80 songs and 6.15 hours and still uses commercial libraries. Public-domain PDMX scores are large enough to matter. Rendering them with one General MIDI bank sounds dry, and short samples do not hold winds or strings.
SPDMX is a listening-guided render of those PDMX scores from UC San Diego. The synth stack is limited to redistributable SoundFonts and MIDI-DDSP weights under Apache 2.0. Clean the MIDI, choose a renderer for each instrument family, then freeze that recipe for the whole corpus.
Track names and program numbers often disagree. A part named violin can be encoded as piano. Aliases on the track name correct the General MIDI program, and tracks with no notes are dropped. That fix only makes the score point at the right patch. It does not make a general-purpose sample bank sound acoustic, which is why the recipe is chosen by ear.
Eight listeners rated realism from 0 to 100 on 10-second clips: ten categories, two stems each. The pilot selects a recipe. It is not a benchmark. Basic is one bank, SGM-V2.01. Varied samples, per category, from FluidR3 GM, GeneralUser GS, Arachno, and Airfont, then adds light reverb, so timbre can vary without Slakh2100's proprietary libraries. MIDI-DDSP, a neural synthesizer trained on the URMP chamber set, takes over eligible monophonic strings, brass, and winds. Polyphonic lines, instruments outside that set, and every other program fall back to FluidSynth. MIDI-DDSP is synthesized at 16 kHz and upsampled to 44.1 kHz so the stems can be summed. The top of the spectrum stays empty. Voice tracks use choir or voice soundfonts. Sung lyrics are out of scope.
The higher mean wins. When basic and varied are close, varied is kept so the timbres do not collapse to one bank. The paper states that this is not a significance test. Drums, organ, and guitar use basic. Piano, voice, mallet, and miscellaneous use varied. Strings and brass use MIDI-DDSP with a basic fallback. Wind uses MIDI-DDSP with a varied fallback.
Matching loudness independently on every stem would erase MIDI velocity balance. Each stem is normalized toward −23 LUFS and peak-limited, then multiplied by its maximum note-on velocity divided by the song maximum. A single anti-clip gain is applied to the sum. The mixture equals the sum of the stems, so a separation model still sees a linear mix.
The figures you can check in the paper are corpus size, not a model score. The release is 253,982 songs, 490,159 mono FLAC stems, and 5,469 song-hours at 44.1 kHz under CC BY 4.0, with MIDI on every song and identifiers back into PDMX. A song-hour is the mix duration counted once, not once per stem.
| Resource | Songs | Mix hours | Type | Open download | No proprietary instruments |
| MUSDB18 | 150 | 10 | recorded | no | n/a |
| MedleyDB | 196 | 13.8 | recorded | no | n/a |
| MoisesDB | 240 | 14.4 | recorded | no | n/a |
| Slakh2100 | 2,100 | 145 | synthetic | yes | no |
| CocoChorales | 240,000 | 1,411 | synthetic | yes | yes |
| EnsembleSet | 80 | 6.15 | synthetic | yes | no |
| SPDMX, all | 253,982 | 5,469 | synthetic | yes | yes |
| SPDMX, multitrack | 88,403 | 2,734 | synthetic | yes | yes |
Songs with at least two stems: 88,403, which is 34.8% of songs and half the hours (2,734 h). Across the full corpus the mean is 1.93 stems per song and the maximum is 66. Single-stem songs: 165,579, or 65.2%. The multitrack slice is already larger than Slakh2100's full 145 hours and larger than CocoChorales' 1,411 hours. CocoChorales still has more songs (240,000 versus 88,403 multitrack songs).
Piano is 57% of stem count and 41% of non-silent time. Non-silent time keeps 0.25-second windows with RMS at least 0.01 (about −40 dBFS), so a lot of piano tracks are short or sparse. Matched-budget separation and transcription checks are only summarized in the paper: in-domain alignment is called usable, and the error numbers live on the project page. A 632-song pack that contains bass, drums, guitar, and piano is marked for comparison with Slakh2100. The comparison itself is not reported.
The usable part is licensed scale. Audio, MIDI, and the release tables are CC BY 4.0. Rendering never goes through a closed sample library, and the same pipeline can be pointed at another symbolic collection. Put 2,734 multitrack hours next to 10 hours in MUSDB18 and 145 in Slakh2100. That is enough synthetic supervision for source separation, stem-conditioned synthesis, transcription, and joint symbolic-audio training. The 65.2% of songs with one stem cannot teach separation. They still have aligned labels for synthesis and transcription.
Listeners only ranked open timbres against each other. There is no published number against commercial plugins or studio stems.
The render is an incremental step. Treat the corpus as large synthetic supervision, then adapt on a small recorded set.
About 65% of songs have a single stem, and piano dominates the inventory. The MIDI comes from engraved scores, so timing and dynamics are quantized rather than performed. Every stem is mono. Eight listeners and two clips per category, scored against other synthetic audio, do not show how close the corpus is to real recordings. MIDI-DDSP remains band-limited after the move from 16 kHz to 44.1 kHz.
Separation and transcription are described as usable in-domain alignment. Budgets, baseline systems, and error rates are absent from the paper, so that claim cannot be checked here. Voice tracks have no lyrics, and a singing model would learn choir pads. The 632-song four-stem pack is set up for a Slakh2100 comparison that was not run in the paper. Public domain clears redistribution. Sample libraries plus monophonic neural synthesis still do not match pop mixes that contain bleed, rooms, and performance noise.