2026-09-25
DR-GEM refits embeddings on high-RE cells, then balanced consensus clustering. It annotates 98% of rare cells on synthetic data versus 0% for standard PCA-Louvain.
The default single-cell workflow is almost always the same stack: PCA, Louvain on a shared nearest-neighbor graph, UMAP, then cluster-level labels. The objective averages error over cells. Abundant types dominate the fit. Rare types barely move the loss.
The bottleneck is not mainly sample size. In the synthetic experiments, covariance among a type's marker genes falls with that type's frequency. 50% is the tipping point: above it, PCA spends more of its budget on within-type variation; below it, those markers nearly vanish from the global covariance. Reconstruction error (RE), the mean squared gap between the input and what the low-dimensional embedding can rebuild, then piles onto the minority cells. Across eight real datasets, cells from a rare type, state, or genetic perturbation are 1.15 to 9.18 times more likely to sit in the worst 10% of RE. On UMAP they scatter and get absorbed into majority clusters.
Off-the-shelf distributionally robust optimization (DRO) does not fix this. KL-DRO and CVaR-DRO do what they advertise: they cut worst-case RE. The rare cells still fail to form a cluster and still get the wrong label. Tilted Fair PCA, even when it is handed the true class labels, drives rare-cell RE down (rank-biserial rrb = -0.95) and still cannot separate them. Fair PCA can equalize error. It cannot make "these cells are their own type" true.
DR-GEM (Distributionally Robust and latent Group-AwarE consensus Machine learning) is a self-supervised wrapper around any dimensionality-reduction plus clustering pair. The default pair is still PCA and Louvain-SNN. Rarity is treated as a signal, not as noise: the method looks at where the first fit failed hardest.
Four phases, the last one optional.
This is not representational self-supervision of the BERT or masked-autoencoder kind. It is closer to hard-example mining and boosting: the model finds its own mistakes and redirects capacity there.
The cleanest numbers are synthetic. Dataset 1 has 21,550 cells and five types; the rarest type has 50 cells. The standard pipeline mislabels all 50. DR-GEM labels 98% correctly, and rare-type RE is no longer higher than the rest (p = 0.78). Swap the base method for polynomial Kernel PCA, LDA, or scVI, and the rare type still sits in high RE until DR-GEM is wrapped around it.
| Method | Rare-cell sensitivity | Balanced accuracy |
| Standard PCA+Louvain | 0% | 0.50 |
| DR-GEM | 98% | 0.990 |
| scCAD | 0% | 0.50 |
| CIARA | 36% | 0.618 |
On Dataset 2, which adds ambient RNA, DR-GEM recovers 88% of rare cells against 0% for the standard pipeline. Contaminated rare cells: 64.7% versus 0%. Restricting to confidence > 0.95 lifts overall accuracy on contaminated cells from 86.9% to 99.0%. When 20% of cells are low-read or batch-shifted, DR-GEM finds 98% and 96% of the rare type; the standard pipeline still labels none of them. With five rare types at once, DR-GEM annotates 90% to 98% of each; the standard pipeline annotates 0%.
SMOTE on Phase I weak labels reaches 100% accuracy and paints the rare population as spuriously heterogeneous. Class-weighted PCA equalizes RE and still cannot form a rare cluster.
Real data follow the same pattern. In CosMx spatial transcriptomics of high-grade serous tubo-ovarian carcinoma, 82,808 non-malignant cells, mast cells are 0.397% (n = 329). The standard pipeline has no mast cluster; DR-GEM labels 85% of them correctly. B-cell accuracy moves from 61% to 78%. On the full SMI dataset the standard pipeline recovers only the four most abundant of seven types; DR-GEM recovers all seven. The paired MERFISH dataset goes from 4/6 to 6/6. Among 185,071 malignant cells from 43 patients, the standard pipeline builds a patient-specific cluster for 34 of 43, and for only 3 of 12 patients who each make up <1% of the data. DR-GEM: 39 of 43 and 8 of 12, plus a spatially distinct subclone (543 cells, 0.29%) that a board-certified pathologist had already marked on matched H&E.
In mouse preoptic hypothalamus MERFISH, pericytes are 0.866% of cells. Standard pipeline: 0%. DR-GEM: 79%. In K562 Perturb-seq, 75,305 non-targeting controls sit next to 371 PTPN1-interference cells (0.49%). DR-GEM reports accuracy 1.00, balanced accuracy 0.94, sensitivity 0.89, against 0.93 / 0.85 / 0.77 for the standard pipeline. Across 14 rare-cell detection tasks DR-GEM wins 11. scCAD is better on PBMCs; CIARA is better on developing thymus. Both of those datasets are easy for every method.
Groups still running Seurat's default PCA plus FindClusters get a drop-in wrapper. The R package is drgem. The use cases are specific: stem and drug-tolerant cells in tumors, transient progenitors in development, under-sampled sgRNAs in Perturb-seq, cell types below 1% in spatial assays. Those populations are supposed to be visible. Average-loss PCA erases them.
This is an incremental method with a sharper conceptual claim. The claim is that "just collect more data" does not fix an objective that treats abundance as importance. KL-DRO, Fair PCA, and rare-cell tools (GiniClust3, scCAD, CIARA) still miss entire classes on synthetic, spatial, and Perturb-seq data. DR-GEM turns those misses into clusters. For labs already training atlases and foundation models, it leaves an untested suggestion: error-guided reweighting might let the same patterns be learned from fewer cells.
Phase II is expensive. Supplementary Data 4 reports 82,808 cells, 500 iterations, 7.23 GB peak RAM, and about 4.2 hours wall time on 10 CPU cores. Large cohorts need parallel runs, the auto-convergence stop, or Phase II on a balanced subset with everyone else projected in. Defaults (beta = 0.05, 500 cells per cluster, t = 0.95) are empirical.
Ground truth on real data is the original papers' curated labels. On the hypothalamus dataset, DR-GEM aligns worse with published inhibitory neurons and Endothelial 2; the authors suggest those labels may be wrong, without an independent assay. Detection of MRPL24i and NRDE2i stays below 50%; they argue the false negatives never actually responded, using perturbation signatures after the fact. That reading can also be a miss. scCAD wins outright on PBMCs, CIARA on thymus. DR-GEM is not an unconditional winner among rare-cell detectors. Nested coarse-then-fine clustering is recommended and not automated. Pairing with single-cell foundation models was only tried with scVI on synthetic data.
In imaging mass cytometry, high RE can also mean the protein panel under-represents a lineage, not that the lineage is rare. Do not throw away low-confidence cells in developmental settings.