Discarding Singletons May Harm Protein Language Model Training
KevinKaichuang · x · 2026-08-18
When training protein language models (PLMs), it is standard practice to discard metagenomic proteins that do not cluster with at least one other protein (singletons).
New research suggests this is likely not the right strategy. The work demonstrates the value of retaining this data and pairs well with findings from ESMC / ESM Fold 2 and Dayhoff, contributing significantly to improving PLM data quality.
More from Research
- Ex-OpenAI researcher claims most neurons are legible now; critics call it hype — anshulkundaje · 2026-10-16
- Physicist pushes back on ML community's acceptance that models may be beyond understanding — tszzl · 2026-10-16
- Survey: AI agent evaluation is getting realistic but still fails to measure safety and reliability — burkov · 2026-10-16
- COVA-SFT dataset trains models to solve text problems with visual chain-of-thought — yoavartzi · 2026-10-16
- USTC paper uses speculative sampling to cut latency in interactive video generation — ustc · 2026-10-16
- Tsinghua's CoTranslator distills reverse CoT to boost 4B model math scores — Tsinghua · 2026-10-16