RITA authors revisit 2022 protein scaling law work as new study shows LLMs rival protein LMs
IgorCarron · x · 2026-10-20
LightOn co-founder Igor Carron dug up his team's 2022 pre-Chinchilla experiments while responding to a new protein scaling study: RITA, autoregressive protein models up to 1.2B parameters trained on 280M+ UniRef-100 sequences — the first systematic study of how capability scales for autoregressive transformers in the protein domain.
- Evaluated on next-amino-acid prediction, zero-shot fitness, and enzyme function prediction, all benefiting from scale
- The shared chart shows LM loss following roughly a 2.0·C^(-0.074) power law in compute (PF-days)
- Carron notes data pruning mattered most on the bio data side, and data size wasn't evaluated back then
- The new study's authors say their key differences: scaling shown across all experiments (not just proteins) and general LLMs matching or beating specialized protein LMs
More from Research
- RFD4-Proteina's 'underrated' docs detail partial diffusion for protein redesign — AllThingsApx · 2026-10-23
- MPI-Sec PhD Thesis on Vulnerability Detection Benchmarks Includes Pre-LLM Classic "Top Score on the Wrong Exam" — mboehme_ · 2026-10-23
- LiveNerf's 29-day data finds no nerfing of Claude Opus 5.5 — TheOnlyVibemaster · 2026-10-23
- New paper makes 9 falsifiable predictions toward a major AI-enabled terror attack by 2027-2028 — StephenLCasper · 2026-10-23
- HumanoidPose recovers humanoid robots' full-body joint states from RGB video alone — cholz · 2026-10-23
- Sophont's Holon-S1 predicts genes and biomarkers from routine pathology slides — iScienceLuvr · 2026-10-23