Adhiraj Ghosh says task-adaptive batch sampling delivered a 3.33x pretraining compute multiplier

pratyushmaini · x · 2026-07-24

Adhiraj Ghosh’s Summer of Data talk covers task-adaptive data curation and adaptive batch sampling for pretraining.

• The core idea is to diversify training batches by concept/task, improving data coverage instead of sampling naively.
• The talk claims this approach produced a 3.33x compute multiplier in pretraining, implying much better efficiency for the same training budget.
• The post links the full talk and frames it as the fifth session in DatologyAI’s Summer of Data seminar series.

Original post →

More from Research

• Researcher flags unresolved data transparency issues in compositionality evals — najoungkim · 2026-10-27
• Separate Forms from Estimators: Why Steer by Difference of Means When You Can Use SGD? — johnhewtt · 2026-10-27
• EMNLP paper probes why some LLM biases survive rewording while others don't — radamihalcea · 2026-10-27
• Datology webinar: Curation Studio matched top data baselines with under 20% compute — josh_wills · 2026-10-27
• KANLab: an interactive browser playground for Kolmogorov-Arnold Networks — Lukas Niehaus · 2026-10-27
• CMU's LLM2Cert synthesizes stability certificates verified via SMT and Lean — CarnegieMellonU · 2026-10-27

Research channel →