Modal Clusters goes GA: instant RDMA-connected GPU nodes billed by the second via one decorator
josh_wills · x · 2026-10-02
Modal has announced general availability of Modal Clusters, a new primitive two years in the making (1.5 years of battle-testing): users spin up globally available, RDMA-connected multi-node clusters instantly with a single @modal.clustered(size=4, rdma=True) decorator, billed by the second.
- The official example defines a training function on 8x B300 GPUs, with cluster.containerips() and containerrank() APIs exposing rank/world-size info for distributed training
- Clusters integrate with existing Modal primitives: checkpoints in Volumes, data via Cloud Bucket Mounts, orchestration via Queues
- Pitch: own your intelligence without owning the hardware, supporting petaFLOP-scale training and serving
Related event: Modal Clusters Reaches GA: Second-Billed RDMA GPU Clusters via a Decorator(2 posts)→
More from Infra
- Mobile AI Factory: agents auto-build on-device SLMs with 19x less GPU time — pmttyji · 2026-11-16
- vLLM serves DeepSeek V4.1-Flash at 954K tokens/s per GPU, $38.96/GPU-hour on GB300 NVL72 — bookwormengr · 2026-11-16
- Colossus 2's 1.2M B200s could generate ~$400B/year in tokens at DeepSeek Flash rates — bookwormengr · 2026-11-16
- BIS issues rule blocking remote access to US AI chips via third-country data centers — teortaxesTex · 2026-11-16
- Qdrant 1.20.1: 50% fewer cold-storage reads, ~2x faster quantization, agent-ready setup — qdrant_engine · 2026-11-16
- One 16GB GPU runs songs, voice, images, 720p video and local RAG — all offline and open source — llama-of-death · 2026-11-16