2026-09-15
Fine-tuning Nucleotide Transformer (250M) on motif-plus-gene pairs yields sequence priors that set the GRN ceiling; GLM-Prior leads 4 of 5 mammalian cell lines.
Gene-regulatory-network (GRN) inference almost always starts from a prior edge matrix: which transcription factors might regulate which genes. Then single-cell expression is used to revise those edges. Priors today come from curated databases such as STRING and TRRUST, or from accessibility-and-motif recipes such as Inferelator-Prior and CellOracle's baseGRN. The databases go missing on understudied species and cell types. The assay-based recipes lock you to matched ATAC-seq and to the assumption that regulation lives near promoters, which is a poor fit for distal enhancers and for moving the prior to another organism.
The bottleneck is the prior, not a fancier downstream solver. GLM-Prior scores each TF–gene pair from DNA sequence alone.
The backbone is the 250-million-parameter Nucleotide Transformer, pretrained on genomes from 850 species. Fine-tuning concatenates a CisBP motif for the TF with the gene-body sequence from a GTF, separated by a classification token. The prepended <cls> embedding goes through a small head (dropout, linear, tanh, dropout, two logits) that predicts whether the pair is a regulatory edge. Positives come from YEASTRACT in yeast and from STRING plus TRRUST in human and mouse; negatives are sampled from the remaining pairs. Class imbalance is handled with negative downsampling, class-weighted cross-entropy, and a DataLoader that balances each batch. A validation sweep picks the F1-maximizing threshold and binarizes the probability matrix.
That matrix then constrains PMF-GRN, a probabilistic matrix factorization of the expression matrix into TF activity and edges. GLM-Prior supplies a logistic-normal prior on edge existence, pinning factors to named TFs. Variational inference returns posterior means and per-edge uncertainty. Training used 4 H100s, effective batch 4096, Adam at \(10^{-5}\), 10 epochs.
Three regimes are tested: same-species train and test, human–mouse transfer, and a single model trained sequentially on human, mouse, and yeast. Held-out gold standards are a literature-curated yeast network and BEELINE cell-line networks for human and mouse. Train and test are disjoint at the level of gene–TF sequence pairs and labels.
Prior AUPRC tracks how many positives and how many TFs the training set actually contains.
| Cell line | GLM-Prior AUPRC | Chance |
| Yeast | 0.02 | 0.01 |
| hESC | 0.19 | 0.15 |
| HepG2 | 0.30 | 0.25 |
| mESC | 0.23 | 0.20 |
| mDC | 0.09 | at chance |
| mHSC | 0.49 | 0.37 |
Yeast trains on 660 positives against 724,179 negatives and barely generalizes. mDC drops from 462 training TFs to 29 at test and stays at chance. mHSC is the largest lift over chance.
Human–mouse transfer holds or slightly improves the single-species numbers: hESC stays at 0.19, HepG2 moves to 0.31. A mammal-trained model on yeast remains at 0.02. The multi-species model mostly copies the single-species scores (yeast 0.02, hESC 0.19, HepG2 0.29, mESC 0.23, mDC 0.09, mHSC 0.49).
Feeding the prior into PMF-GRN never lowers AUPRC: HepG2 0.30→0.38, mESC 0.23→0.27, mHSC 0.49→0.52; yeast and hESC are unchanged. Edge edits are tiny, with additions from 0.03% in yeast to 0.54% in mHSC, and almost no deletions. A weak prior is not rescued by expression.
Against accessibility priors, GLM-Prior is at or above chance in all six lines and clearly above in five. Inferelator-Prior clears chance in two lines, CellOracle in three. Yeast is the exception: CellOracle 0.05 versus GLM-Prior 0.02. In HepG2, CellOracle 0.36 beats GLM-Prior 0.30; the paper traces that to a BEELINE reference built from ENCODE ChIP-seq, which matches motif-in-accessible-peak linking. GLM-Prior is the best prior in the other four mammalian lines (hESC, mESC, mDC, mHSC). Across nine prior × algorithm pairings, the winner in each line almost always uses that line's strongest prior. No inference method wins everywhere.
If the working habit is to swap Inferelator, CellOracle, or a matrix factorization, this paper reverses the priority: get the prior right, then let the algorithm edit under that ceiling. GLM-Prior does not need matched ATAC. A reference genome and a motif library are enough to emit a transferable scaffold, and human-to-mouse transfer actually works. Yeast and mDC show that a language model does not invent supervision that is not there. Incremental, with a clear reallocation of effort from the solver to the sequence prior.
Many negatives are untested edges, not confirmed absences, and downsampling can drop false negatives. Reference networks are incomplete, so AUPRC moves with how the gold standard is built. CellOracle winning on HepG2 is a reminder that the reference construction can reorder methods. The input is a motif plus a gene body, not enhancer–promoter pairs with 3D genome evidence; distal regulation is inferred from sequence and is not checked with Hi-C. Only six cell lines. Close-mammal transfer holds; transfer into yeast does not.