Bittensor IOTA cuts decentralized training merge time 5x to 6.7 min on 16B
bittingthembits · x · 2026-10-22
Bittensor subnet IOTA (SN9) posted proof-of-work numbers for decentralized training optimization: median per-round merge time on a 16B model dropped from 31.8 min to 6.7 min (5x faster, a 79% cut).
Context: in decentralized training, every merge stalls the whole swarm waiting on the slowest machine. Results with the same 24 hours and 140 machines:
- 19% more tokens trained, loss 4.00 vs 4.17
- Addressing concerns that 4-bit compression and skipping stragglers hurt quality: at equal token counts loss is 4.04 vs 4.05, curves nearly identical — the speedup is pure systems work with no quality loss
- Setup: 138 workers, 10 pipeline stages, 14 replicas; training is also 20% cheaper
More from Infra
- C multithreading experiment: padding counters to 64 bytes yields 3-4x speedup via false sharing fix — cneuralnetwork · 2026-10-23
- Two £300 Snapdragon mini PCs, no GPU: 46 tok/s local Qwen3.8 Flash Next with custom ToRc engine — WorldlyAd7946 · 2026-10-22
- ai& runs MiniMax H3 across NVIDIA, AMD and Tenstorrent, undercutting official pricing — DavidBennett__ · 2026-10-22
- With PS5 and Switch 2 jailbreaks taking off, could consoles run local LLMs? — cute_spider · 2026-10-22
- Jensen Huang defends neocloud financing: high borrowing costs are a side effect of huge AI demand, not a business flaw — rohanpaul_ai · 2026-10-22
- PHP, Node.js, GitLab and LiteLLM hit critical security incidents the same day — alex_verem · 2026-10-22