Universal Data-Efficient Pretraining Claims 56x Gains on Molecules, 24x on DNA and Crystals
madhavsinghal_ · x · 2026-11-19
A team with qlabs proposes a universal recipe for data-efficient pretraining that works across domains: up to 56x data efficiency on molecules and 24x on DNA and crystal structures versus SOTA autoregressive recipes. After fine-tuning, the models beat ones pretrained on hundreds of times more data.
The authors argue that since scaling in nearly every domain is bottlenecked by data, data efficiency is the most important algorithmic problem in AI — and this could fundamentally change data-constrained scaling across all domains. Results shown on DNA, molecules, and crystalline structures.
More from Research
- Meta researcher jokes his AI agents 'just cheered and agreed' during paper work — ziv_ravid · 2026-11-19
- NNterp 2.0 launches: run the same NNsight interpretability experiment across ~all LLMs — gsarti_ · 2026-11-19
- CONGA compiler exploits Gaussian block geometry for Bayesian inference, 12.5x ESS/s gain for NUTS — Víctor Gallego · 2026-11-19
- GPT 6.2 Astra shows surprisingly strong zero-shot crystal generation, a year after the task looked hopeless — xennygrimmato_ · 2026-11-19
- Fine-tuned World Action Model Dream-1 Jumps 159%, Beats Frontier LLMs on Robot Tasks — paigeinsf · 2026-11-19
- AI makes 'living papers' nearly free to build, but their durability is doubtful — _akpiper · 2026-11-19