Low perplexity can be pure memorization: caveat for the protein LM debate

proteinrosh · x · 2026-11-18

proteinrosh adds a key caveat: you can also reach 2.2 perplexity by training a giant model without clustering — the model effectively memorizes the data, and its representations aren't useful for structure prediction. Perplexity alone doesn't prove structural understanding; it must be validated with structure prediction tasks.

Related event: Protein Model Perplexity Debate: Does 2.2 Mean Structural Understanding?(3 posts)→

Original post →

More from Research

Research channel →