2026-07-31
Structure conditioning holds protein-design models' taxonomic bias under 3% versus ~20% for sequence-only models; the bias tracks a few biophysical axes and fine-tuning flips it.
Feed the same protein backbone to different design models and you get very different amino-acid sequences. That divergence is not noise; it is a systematic preference baked in by training data. Ding and Steinhardt showed in 2024 that protein language models like ESM2 favor sequences from model organisms such as E. coli and humans, while penalizing functionally equivalent proteins from underrepresented species.
The open question is whether this taxonomic bias is unique to sequence-only models, or whether structure-conditioned models (ProteinMPNN, ESM-IF, which read 3D coordinates to recover a sequence) inherit it too. The practical stakes are sharper: when a design model scores a sequence low, you cannot tell whether the sequence is structurally unfit or merely comes from a species the model rarely saw. Those two reasons collapse into one score, and that score decides where wet-lab effort goes.
The authors assembled 7,843 proteins across 144 species spanning all three domains of life, and scored them with AlphaFold2-predicted structures (the PDB was too thin on archaea and extremophile bacteria). They ran six models covering three training paradigms: structure-conditioned (ProteinMPNN on PDB, ESM-IF on AlphaFold DB), hybrid (MIF on CATH; MIF-ST adds CARP sequence embeddings from 42 million sequences), and sequence-only (ESM2-15B, CARP-640M).
Three tools break the bias apart. Variance decomposition peels the total variation in model scores into protein family, structural features, quality, and sequence properties, leaving the slice attributable to species identity (additional R²): the residual taxonomic bias that survives after every measurable property is controlled for. An ELO rating system borrowed from chess runs pairwise species matchups inside each protein family. A PCA over 11 biophysical features plus generalized additive models (GAMs) draws each model's preference landscape.
The cleanest piece of the design is the MIF versus MIF-ST contrast. The two share near-identical architecture and both train on the taxonomically balanced CATH set; the only difference is that MIF-ST injects CARP's pre-trained sequence embeddings. That is an isolated variable for measuring how much bias the embeddings drag back in.
Variance decomposition gives the cleanest comparison. After controlling for protein family and biophysical covariates, structure-conditioned models leave under 3% of score variance attributable to species (ProteinMPNN 2.21%, ESM-IF 1.12%, MIF 2.80%), close to measurement noise. Sequence-only models still hold roughly 20% (ESM2-15B 19.47%, CARP-640M 21.42%).
| Model | Paradigm | Residual species variance (Add. R²) |
| ESM-IF | Structure | 1.12% |
| ProteinMPNN | Structure | 2.21% |
| MIF | Hybrid | 2.80% |
| MIF-ST | Hybrid + seq transfer | 11.27% |
| ESM2-15B | Sequence | 19.47% |
| CARP-640M | Sequence | 21.42% |
So a low score from ProteinMPNN almost always means genuine physicochemical incompatibility with the backbone; a low score from ESM2 could mean poor quality or an underrepresented origin, and those two signals are confounded.
The bias direction also inverts between paradigms. Structure-conditioned models consistently favor archaea and bacteria: ProteinMPNN rates archaea at 1693 and eukaryota at only 1247. Sequence-only models flip the order and favor eukaryota: CARP scores eukaryota 1639 and archaea just 961, a 678-point gap. The two preferences point opposite ways, not in degree.
The isolated-variable result nails the contamination from sequence embeddings. MIF and MIF-ST share architecture and training data, yet adding CARP embeddings pushes MIF-ST's residual species variance from 2.80% to 11.27% and turns its species preferences to track CARP (ρ=0.67) more than its own twin MIF (ρ=-0.17). Pre-trained embeddings from a taxonomically skewed database override deliberately balanced structural training.
PCA compresses these preferences to two dimensions that explain 51.4% of the variance. PC1 tracks compactness and protein size (compact proteins at the negative end, large extended proteins at the positive end); PC2 tracks hydrophobicity and stability. Structure-conditioned models cluster in the compact, low-instability region, and archaeal proteins happen to be naturally compact while eukaryotic proteins are larger and more extended. The archaea preference is really a by-product of preferring compact proteins. The bias is low-dimensional and interpretable, not a high-dimensional black box.
For anyone doing protein design, this is a basis for model choice. Designing a compact, thermostable protein? ProteinMPNN's preferences align. Designing membrane proteins or intrinsically disordered regions? The same preferences become obstacles. The authors cite Derry et al.: models trained only on X-ray structures systematically downweight lysine, not because lysine destabilizes proteins but because its conformational entropy hurts crystallization. The model learned "easy to crystallize" as a proxy for "good protein." There is also a counterintuitive reminder: ProteinMPNN ranks lowest on the ProteinGym mutational-effect benchmark (Spearman 0.26) yet may be the better choice for design. Benchmark rank and design preference are different things.
For the wider AI community, this is a clean case study of generative-model bias: it traces to training data, it is quantifiable, it is low-dimensional, and it can be redirected on demand.
The authors concede two points: correcting several bias dimensions at once through fine-tuning may be impractical, and how far the approach generalizes to other preference types is unverified. They also admit that scoring patterns alone cannot tell whether ProteinMPNN's high archaea scores reflect designability, training composition, or genuine fitness advantage.
Two more concerns stand out. First, the entire dataset uses AlphaFold2-predicted structures rather than experimental ones, and AlphaFold itself was trained on the skewed PDB, so the predictions already embed the previous generation's assumptions. The authors flag this feedback-loop risk in the discussion, but their own measurements rest on predicted structures, meaning the "clean" residual variance could partly come from two layers of bias canceling. Second, the design experiment uses only 16 template proteins, and the authors note that single-protein evaluations mislead.