Distributed inference on idle Macs hits 99.87% uptime because inference jobs are light-state
gajesh · x · 2026-10-20
Responding to questions about @darkbloomai's reliability during power outages, gajesh shares a key insight on distributed inference:
- Inference jobs run for minutes and carry very light state—worst case, you lose a tiny bit of progress and resume elsewhere
- This has plagued decentralized general-purpose networks, but inference is a light-state, transferable workload that fits distribution naturally
- Quoting @cattttang: datacenters are assumed to need high reliability, but if your workload tolerates 95% or 90% uptime, you can migrate between sites and exploit cheaper underutilized capacity
- Result: 99.87% uptime on a distributed network of idle Mac machines
More from Infra
- Cadence announces sovereign AI cloud for European chip designers — karlfreund · 2026-10-23
- SGLang Deep-Dive Blog Series Breaks Down KV Cache, Posts up to 15.5x Throughput Gains — BanghuaZ · 2026-10-23
- DynaLM: Open-Source CPU-First LLM Inference Engine Seeks Feedback — Unlikely_Safety_7456 · 2026-10-23
- Capital Is Now the AI Buildout Bottleneck, Says Analyst Citing Jensen Huang — JOBhakdi · 2026-10-23
- Cerebras explains how it deploys trillion-parameter models on wafer-scale chips — bookwormengr · 2026-10-23
- IFM open-sources RL360, the agentic RL infra behind K2 Horizon's training — rupspace · 2026-10-23