Training run at ~$11/b tokens with insane MFU, datasets streamed over p2p backbone
jon_durbin · x · 2026-09-11
The author shares a dashboard and details of a training run: roughly $11/b tokens, with MFU described as "insane" and inference also strong.
Technical detail: GPU node bandwidth carries both training and dataset streaming — nodes don't predownload datasets; data streams from p2p backbone/syncer nodes with cursor failover for distribution and failover handling. Some nodes came from lium.io.
More from Infra
- SpaceX lands another AI compute hosting deal worth $1.1B per month — NicoVerderosa · 2026-09-11
- L3Harris says fine-tuned open-source models beat frontier AI in 48 hours at 95% lower cost — eliano · 2026-09-11
- Qwen3-TTS 1.7B hits 1.6x real-time voice cloning on CPU via llama.cpp — alexcovo_eth · 2026-09-11
- NVIDIA ships NVFP4-quantized Qwen3.8-27B, trending on Hugging Face — nvidia · 2026-09-11
- Qwen 3.8 125B-A6B runs 15.3% faster on Mac via speculative decoding on mlx.fast — TheMoonMidas · 2026-09-11
- Olam Labs CEO: only compute and data remain as bottlenecks to AGI — garrytan · 2026-09-11