Weight teleportation open-sourced: cross-datacenter RL weight updates under 0.23s
gharik · x · 2026-11-03
The team behind gharik's account developed and open-sourced "weight teleportation", tackling the slow and expensive trainer-to-sampler weight updates that bottleneck large-scale RL — even across datacenters.
Key numbers:
- Sampler pause time under 0.23s, with 3s non-blocking compression + prepare
- Faster than existing intra-cluster update techniques at internet speeds; the bottleneck becomes the GPU's decompression ASIC
Core techniques: massively parallel compression of model weight deltas on GPUs, a heterogeneous decompression and patching pipeline, plus hardware-co-designed optimizations and overlap. Open source.
More from Infra
- Gratis launches: run open-source AI models locally in your browser with zero setup — markjeffrey · 2026-11-03
- Weight teleportation cuts RL sampler pauses below 0.23s at internet speeds — gharik · 2026-11-03
- NVIDIA details Cosmos Framework: world model training, sim recipes, and NIM deployment — NVIDIAAI · 2026-11-03
- Anthropic's message threads cut TTFT 35% on 1M-token agent sessions — ClaudeDevs · 2026-11-03
- SPC NYC hosts stacked panel on compute trading with Jump Trading and Cantor leads — ditzikow · 2026-11-03
- Workers IO launches to automate verification, simulating millions of fault-injected production scenarios for AI-written code — ycombinator · 2026-11-03