Learning Where to Split: Dynamic Band Splitting for Audio Source Separation
Bingzhe Zhang, Kai Li, Xiaolin Hu
cs.SD
2026-10-10
Dynamic Band Splitting moves frequency boundaries with each mixture under a fixed band-token budget, and DBS-BSRNN raises average SI-SDR on DnR v2 from 7.69 to 9.07 dB.
Source separation pulls music, speech, and effects out of one recording. Band-split models start from a short-time Fourier transform (STFT, a time-frequency grid), group the frequency bins into bands, and hand the separator one token per band. BSRNN, BS-RoFormer, Mel-Band RoFormer, and TIGER all work this way. The token sequence is much shorter than the STFT frequency axis, which is what keeps the temporal and cross-band stack affordable.
The boundaries are written by hand and reused for every mixture. A wide band can squash a vocal harmonic and a metallic hit into the same token. A finely sliced band can sit on a stretch of spectrum with almost no energy. More bands buy resolution and lengthen the sequence. With the token budget held fixed, the resolution has to follow the mixture.
Dynamic Band Splitting (DBS) chooses the boundaries from the mixture and locks the number of bands. The frequency axis is first cut into fixed blocks, so lows and highs each keep a quota of tokens. Only the cuts inside a block move. The experiments use four blocks. DnR v2 and MUSDB18-HQ use 1025 STFT bins and 128 band tokens, 32 in each block. EchoSet is 16 kHz audio, with 321 bins, 67 tokens, and block budgets of 40, 10, 8, and 9.
Inside a block, real and imaginary parts pass through temporal convolutions, and a bidirectional LSTM builds a context vector at each bin. A larger gap between neighboring contexts looks more like a boundary. A second term uses the magnitude mean, the peak-to-mean ratio, and the standard-deviation-to-mean ratio. The gap between neighboring statistics is added with a fixed coefficient of 0.15. A learned scale and a position bias go through a sigmoid to produce a boundary probability. The first bin of the block is always a boundary. Hard Top-K keeps the highest-scoring cuts for the remaining slots. Bands stay contiguous, the count matches the budget, and the widths may differ.
The separator still expects a fixed feature size. Each bin is encoded, scored from its time-averaged feature, and pooled with those weights into one vector. Length and dimension stay the same. What changes is which bins share a token. The temporal and cross-band cores of all four backbones are left in place.
The mask has to be drawn on every STFT bin. Each separated token is blended with the previous band using that band's average boundary probability. A higher probability keeps more of the current token. The smoothed tokens are copied back onto the original grid, then a residual block along frequency lets neighboring bins mix. A head shared across frequency predicts a complex mask for each source. The mask multiplies the mixture spectrogram, and an inverse STFT returns the waveform.
Hard Top-K is discrete in the forward pass. A straight-through estimator keeps that hard partition forward and sends the backward pass through the local difference between a boundary being present and absent. A density penalty pulls each block's mean boundary probability toward a target compression ratio. On DnR v2 the weight is 0.01. The band count that actually ships is still J.
DnR v2 separates music, speech, and effects, scored with SI-SDR. Trained from scratch at the same 128 bands, static BSRNN averages 7.69 dB and DBS-BSRNN 9.07 dB, a gain of 1.38 dB. Music moves from 4.83 to 6.48, speech from 14.87 to 15.01, and effects from 3.38 to 5.73. TIGER moves from 8.74 to 10.13 dB. The abstract's 29.0% relative gain uses the published BSRNN average of 7.03 dB as the denominator, not this matched 128-band run.
| Method | Avg. SI-SDR | Params | Compute |
| BSRNN (128) | 7.69 dB | 35.5M | 76.6 G/s |
| DBS-BSRNN | 9.07 dB | 4.7M | 124.0 G/s |
| TIGER (128) | 8.74 dB | 1.4M | 20.4 G/s |
| DBS-TIGER | 10.13 dB | 1.0M | 31.6 G/s |
DBS-BSRNN drops from 35.5M to 4.7M parameters and rises from 76.6 to 124.0 G MACs per second. The boundary predictor and the full-resolution decoder put that compute back.
On MUSDB18-HQ one network emits vocals, drums, bass, and other, trained only on the official 100 tracks, with no extra audio. Average SDR goes from 5.09 to 5.43 dB for BSRNN, 4.50 to 6.34 dB for BS-RoFormer, and 7.25 to 8.07 dB for MelRoFormer. BS-RoFormer parameters fall from 378.5M to 33.9M, with MACs nearly flat at 292.7 versus 290.5 G/s. MelRoFormer MACs rise from 171.7 to 252.9 G/s. Absolute scores sit below public systems trained with extra data. The comparison holds only inside this setup.
EchoSet is two-speaker speech with noise and reverberation. DBS-BSRNN reaches SDRi 13.21 dB and SI-SDRi 12.66 dB, against published BSRNN at 12.75 and 12.23, gains of 0.46 and 0.43 dB. SDRi is the improvement over using the mixture itself as the estimate. DBS-TIGER reaches 14.55 and 14.04, against TIGER at 14.22 and 13.73, gains of 0.33 and 0.31 dB.
On 64 DnR test mixtures, the middle 3-second window produced a different four-block partition every time. A second pass on the same input reproduced the boundaries. Across mixtures, high-frequency block B4 replaced 46.13% of its internal boundaries. B2 replaced 7.20%. Adjacent windows differ. Inside one window the partition is frozen.
Using one block instead of four drops average SI-SDR to 6.30 dB, and the cuts pile up at low frequency. Removing the neighboring-magnitude transition drops the score to 4.09 dB. Removing frequency refinement drops it to 3.92 dB. Scoring boundaries from magnitude statistics alone drops it to 6.93 dB. On the trained DBS-BSRNN, swapping the learned cuts for the BSRNN layout at inference lowers 9.07 dB to 8.62 dB. Uniform cuts lower it to 7.80 dB. Of the 1.38 dB gap versus a static model trained from scratch, boundary placement alone accounts for 0.45 dB.
The separator sequence does not have to grow for the resolution to follow the mixture. On the same BSRNN and the same 128 bands, a separately trained uniform partition scores 8.00 dB on DnR, Mel bands score 5.87 dB, and DBS scores 9.07 dB. Music and effects land in different places, and one fixed grid covers them poorly.
Dropped onto an existing band-split backbone, DBS replaces the band interface and the mask head. The temporal stack stays. Per-band heads become one frequency-shared head, so the parameter count usually falls. Compute does not follow. Against the published BSRNN on EchoSet, DBS-BSRNN goes from 98.7 to 38.1 G/s. Against the matched 128-band BSRNN on DnR, it goes from 76.6 to 124.0 G/s.
Cinematic separation gains about 1.4 dB. Four-stem music gains 0.35 to 1.84 dB. Two-speaker speech gains under 0.5 dB. Speech spectra are more stable, so a fixed cut was already less wrong. The gain shows up where the spectrum keeps changing.
The band count is fixed, and each inference window gets one partition. On DnR the window is 3 seconds with a 1.5-second hop, so a change inside the window does not move the cuts. When the model could learn the count, it pushed every block to the largest budget allowed. A variable budget needs its own complexity penalty. Events shorter than the window never get their own partition.
The 1.38 dB is not all boundary placement. With the weights frozen, switching to the BSRNN layout costs 0.45 dB. The rest is mixed in with the shared mask head, the weighted pooling, and the frequency refinement. The MelRoFormer pair, as the paper states, compares entire band interfaces. BS-RoFormer shrinking from 378.5M to 33.9M parameters is a much larger change than sliding a few cuts.
The unconstrained learnable filterbank averages -11.72 dB, which is a weak control. Tested on its own, that analysis-synthesis path rebuilds a probe at an SNR of -42.5 dB with correlation 0.03, and 923 of 1025 filters peak below 1 kHz. The run rules out a broken front end. It does not say how DBS would compare with an adaptive front end that actually reconstructs.
MUSDB absolute SDR is not comparable to public models trained with extra data and per-stem networks. EchoSet is compared with published numbers, not a static model retrained on the same schedule. No listening test is reported.