Meta researcher's training trick: drop a batch proportion instead of per-sample to keep GPU efficiency
TimDarcet · x · 2026-09-09
Meta researcher TimDarcet shares a practical training trick: instead of dropping each sample with probability p, drop a proportion p of each batch. This keeps tensor shapes constant and preserves GPU efficiency. He says this is how it's done in Meta's open-source capi vision training library (see model.py), and notes that if everything stays on GPU it compiles through perfectly.
Related event: Meta researcher shares batch-ratio token dropping training trick(3 posts)→
More from Research
- HarnessMM cuts agent tokens per task by up to 88% while raising success rates — omarsar0 · 2026-10-26
- New Blog Breaks Black-Box Distillation Into Five Practical Steps — chrisrohlf · 2026-10-26
- DetectorScope turns black-box AI text detectors into auditable ones with sparse autoencoders — _akhaliq · 2026-10-26
- OPST paper shows LLMs self-improve with zero human data, guided only by topic descriptions — _akhaliq · 2026-10-26
- SJTU's VaM lets robots learn online from mistakes, 95%+ success over 4-hour autonomous runs — chris_j_paxton · 2026-10-26
- Hugging Face uses small open models to reverse engineer a game binary byte-for-byte — yb2698 · 2026-10-26