Drop a fixed batch proportion instead of per-sample tokens: capi author shares training trick
giffmana · x · 2026-09-09
- Tim Darcet suggests improving token dropping: replace "each sample is dropped with probability p" with "a proportion p of the batch is dropped", yielding constant shapes, good efficiency, and perfect compilation when everything runs on GPU.
- giffmana notes this is the same permute+slice trick as in tokendrop, but cautions it only works for fixed example sizes — increasingly rare in both vision and language models where you may want to fully keep or drop layers per example. Implementation referenced in the capi repo.
Related event: Meta researcher shares batch-ratio token dropping training trick(3 posts)→
More from Infra
- Same budget: MacBook Air + 48GB VRAM desktop rig vs 64GB MacBook Pro for local LLMs — gappyvalley · 2026-09-09
- NVIDIA launches CUDA Rust: two tracks to write GPU kernels in Rust — ducha_aiki · 2026-09-09
- OpenAI and Samsung are jointly developing and producing next-gen AI chips — rohanpaul_ai · 2026-09-09
- Hetzner VPS bills jump 543% in 2 years, from €5.30 to €34.09 per month — ThePeterMick · 2026-09-09
- Harbor OSS now processes 80T tokens per week, edging past OpenRouter's 70T — simonguozirui · 2026-09-09
- Supabase's Multigres hits 100% pass-rate on major Postgres test suites for PB-scale — dshukertjr · 2026-09-09