Universal Data-Efficient Pretraining Claims 56x Gains on Molecules, 24x on DNA and Crystals

madhavsinghal_ · x · 2026-11-19

A team with qlabs proposes a universal recipe for data-efficient pretraining that works across domains: up to 56x data efficiency on molecules and 24x on DNA and crystal structures versus SOTA autoregressive recipes. After fine-tuning, the models beat ones pretrained on hundreds of times more data.

The authors argue that since scaling in nearly every domain is bottlenecked by data, data efficiency is the most important algorithmic problem in AI — and this could fundamentally change data-constrained scaling across all domains. Results shown on DNA, molecules, and crystalline structures.

Original post →

More from Research

Research channel →