Training Music Sample Identification Models on Real Sample Pairs
R. Oguz Araz, Joan Serrà, Xavier Lizarraga-Seijas, Emilio Molina, Xavier Serra, Yuki Mitsufuji, Dmitry Bogdanov
cs.SD
2026-09-18
SIE trains on real WhoSampled pairs, reaching 0.758 and 0.701 mAP on Sample100 and SamplePairs (26% and 56% above SampleID); 5% real pairs match SampleID on the large test set.
Sample identification asks a retrieval system to find the source track that a later recording sampled and musically transformed. The overlap is often a short excerpt, then time-stretched, pitch-shifted, chopped, or scratched. That is a different job from track identification, which matches near-copies of the same recording, and from version identification, which matches covers of a whole song.
The task dates to 2013, but large labeled sets did not exist, so recent models synthesize pairs from multi-stem audio: stretch and transpose a stem, then remix it. Those pipelines do not reproduce chopping or scratching. WhoSampled130K now supplies 79,111 real sample pairs among 114,721 commercially released tracks. What was missing was a training recipe that actually uses them.
SIE (SI Embeddings) grafts SampleID's sample-identification front-end onto the Fish architecture and training recipe.
Training pairs come from WhoSampled130K: 79,111 pairs, 114,721 tracks, all resampled to 16 kHz mono. Annotations mark the start of a sample at 1-second resolution, not the end, so duration is unknown. Each positive pair is built from one annotated pair by drawing a random timestamp per track, jittering it independently in [-2.5, 2.5] s, and loading 10 s of audio. Peak normalization to a target in [-30, 15] dBFS adds clipping when the target is positive.
The encoder sees a 258-bin variable-Q transform from 27.5 Hz at 36 bins per octave. Training uses 230 frames at 40 Hz, which is 6 s. The back-end is ResNet50-IBN with GeM pooling and a projection head (batch norm, linear, L2), producing 1024-dimensional embeddings, half the size of the baselines.
Three augmentation chains fire independently at probability 0.25:
The loss is triplet with Euclidean distance and margin 0.3. Each step builds a batch from 394 real pairs and mines the hardest in-batch negative. Adam, no weight decay, learning rate 3×10^{-4} cosine-annealed to 10^{-6} over 20k steps. Training takes 25 hours on one L40S with FP16 mixed precision.
To separate data from recipe, they retrain SampleID on the same real pairs using the released code, applying stretch and transpose at probability 0.25 instead of always. That baseline, SampleID-R, is trained to convergence for 38k steps.
On the two published small benchmarks, SIE is ahead of every reported number. Sample100 has 75 queries and 104 pairs, all hip-hop. SamplePairs has 100 queries and 100 pairs, mixed genre.
| Method | Training pairs | Dim. | Sample100 mAP | SamplePairs mAP |
| Van Balen et al. (rules) | none | - | 0.390 | not reported |
| Cheston et al. | synthetic | 2048 | 0.441 | not reported |
| SampleID | synthetic | 2048 | 0.603 | 0.450 |
| SampleID-R | real | 2048 | 0.651 | 0.511 |
| SIE | real | 1024 | 0.758 | 0.701 |
Relative to SampleID, that is +26% and +56% mAP. Swapping data (SampleID-R vs SampleID) adds 0.048 / 0.061 mAP. Swapping architecture and recipe (SIE vs SampleID-R) adds 0.107 / 0.190. The data swap already sets a new state of the art; the recipe still accounts for more of the gap. Mean normalized average rank moves the same way: SIE reaches 3.9 and 2.4 on the two benchmarks, against SampleID at 7.6 and 4.7.
On WhoSampled130K's test split (8,463 pairs, 10,444 tracks, about 10k queries) the ranking is unchanged:
| Method | mAP | mNAR (lower is better) |
| SampleID | 0.226 ± 0.007 | 19.4 ± 0.5 |
| SampleID-R | 0.299 ± 0.008 | 14.3 ± 0.4 |
| SIE | 0.389 ± 0.009 | 13.0 ± 0.4 |
| NMFP (track ID) | 0.101 ± 0.005 | 35.4 ± 0.6 |
| CLEWS (version ID) | 0.181 ± 0.007 | 32.4 ± 0.7 |
| Fish (track + version) | 0.207 ± 0.007 | 29.9 ± 0.6 |
SampleID, trained on synthetic pairs, beats Fish (trained for versions and tracks, not samples) by only 0.019 mAP. Real pairs open that gap.
SIE trained on 5% of the real pairs already matches SampleID's mAP on this large test set, even though SampleID saw an effectively unbounded stream of synthetic pairs. Adding more real pairs still helps; the curve has not saturated.
Ablations are consistent. Signal manipulation plus degradation: 0.389 mAP. Manipulation only: 0.365. Degradation only: 0.334. Neither: 0.320. Triplet beats the best NT-Xent setting by 0.057 mAP (0.389 vs 0.332). Segment lengths of 5, 6, and 8 s sit inside the confidence intervals.
Rights monitoring, sample tracing, and sample-aware discovery all needed a recipe that works on real pairs, not a stem run through a synthesizer. This paper supplies the first fully supervised one, with code and checkpoints.
The practical takeaways are blunt. Use real pairs when you have them, even a few percent of a synthetic corpus. Do not treat the recipe as a footnote: triplet loss, Fish-style degradation, and the SampleID front-end together move more than the data swap. 1024-d embeddings are enough and cut index size in half.
This is a transfer of working parts from neighboring retrieval tasks, not a new backbone. Incremental, and the direction is no longer ambiguous.
Sample100 and SamplePairs have too few queries for confidence intervals to separate systems; the paper reports point estimates only. The WhoSampled130K test set contains 71% of Sample100 tracks and 50% of SamplePairs tracks. Training tracks do not overlap those benchmarks by construction, and no training track is sample-paired with them, but the small-benchmark numbers can still look optimistic.
Labels give start times, not durations, so the model never sees how long a sample lasts. The augmentations are still stretch, transpose, and degradation. Chopping and scratching, the transformations used to argue against synthetic pipelines, are not modeled explicitly. Part of SIE's gain is the Fish recipe, not the phrase "real pairs" alone.
Performance has not saturated at 100% of the training set. The reverse question is open: if a synthetic pipeline imitated producers more closely, would it transfer at scale. Bhattacharjee et al. are left out of the comparison because their first-stage embeddings are not meant to stand alone, and because public reproductions of their numbers have been disputed.