ESM researcher: covariation in data made LLMs and AlphaFold possible — other modalities need it too
sokrypton · x · 2026-11-04
Protein language model researcher sokrypton argues that the success of LLMs on text/code and of PLMs/AlphaFold on proteins fundamentally rests on covariation patterns extracted from data. To advance other modalities, he says, we must collect equivalent data — under natural or artificial selection — from which covariation can be extracted.
He adds that the key challenge is finding fast, cheap ways to gather detailed information from the huge biodiversity of life before it goes extinct. The takeaway: the bottleneck for generalizing to new modalities may lie in data collection rather than architecture.
More from AGI Musings
- London Autonomous Organization Summit: ARIA co-hosts event on securing the agentic web — weballergy · 2026-11-05
- Kevin Roose interviewed 150 AI insiders for The AGI Chronicles, fact-checked with a council of Claudes — every · 2026-11-05
- AutoDNA: AI agents run multi-step molecular biology experiments with minimal human input — james_y_zou · 2026-11-05
- Amazon blocks Meta's Muse shopping agent over ad revenue threat — XIFAQ · 2026-11-05
- bfl_ai co-founder explains world models: intelligence learned from natural signals — shiringhaffary · 2026-11-05
- Kai-Fu Lee in TIME: why almost every corporate AI transformation fails — kaifulee · 2026-11-05