Training run at ~$11/b tokens with insane MFU, datasets streamed over p2p backbone
jon_durbin · x · 2026-09-11
The author shares a dashboard and details of a training run: roughly $11/b tokens, with MFU described as "insane" and inference also strong.
Technical detail: GPU node bandwidth carries both training and dataset streaming — nodes don't predownload datasets; data streams from p2p backbone/syncer nodes with cursor failover for distribution and failover handling. Some nodes came from lium.io.
More from Infra
- Open-source Latch compiles PyTorch into training megakernels, 2.75x faster on H100 — sguada · 2026-10-27
- DeepSpeed shows AutoEP/AutoTP/AutoSP parallelism gains, hosts office hours — PyTorch · 2026-10-27
- Running a local 9B as a coding agent: brief the big model to plan, let the small model execute — Electrical_Ninja3805 · 2026-10-27
- Cloudflare Durable Objects validated as top architecture for AI agent WebSocket servers — tristanbob · 2026-10-27
- Gary Marcus: With Q3 data added, Anthropic's compute-efficiency growth shows deceleration — GaryMarcus · 2026-10-27
- SpaceX locks in $40B GPU financing deal with Apollo, the largest ever committed — TechCrunch AI · 2026-10-27