antirez prefetches MoE experts via router trick to run Kimi K3.1 on a 128GB machine
antirez · x · 2026-11-05
antirez shares a trick for streaming Kimi K3.1 on small (128GB) machines: computing the next layer's MoE router using the previous layer's residual almost always correctly predicts the top-4 experts, allowing SSD prefetch hints to hide load latency. He adds the same approach should work for DS4.1F.
More from Infra
- LFM 3 1.7B runs on-device on folding iPhone at a stable 60+ tokens/s via MLX — helloiamleonie · 2026-11-05
- Mach 2 Medium sets new Pareto frontier for Qwen 3.8 compression: 1.5% loss at 26.7GB, beating Unsloth — pbaylies · 2026-11-05
- Redditor finds Tiel-Coder-35B MoE runs faster than Qwen 3.8 locally with 256K context — zxspectrun · 2026-11-05
- 63% of planned US data centers sit in states whose governors pledged bans or pauses — Catholic_Indian10 · 2026-11-05
- OpenAI's Stargate to consume up to 40% of global DRAM output via Samsung/SK hynix deal — TheZachMueller · 2026-11-05
- Mooncake author presents fleet-wide elastic KV cache across Cloudflare on Kastan Day — michellechen · 2026-11-05