A 2007 survey on collective communication still helps when learning NCCL
abhi9u · x · 2026-07-26
A 2007 survey paper on collective communication is still useful background for anyone learning NCCL and multi-GPU programming.
- The paper collects the main techniques behind collective ops such as scatter, gather, and reduce-scatter.
- It focuses on how these algorithms are implemented and on their cost model, which helps when reasoning about performance even if NCCL hides the details.
- The survey was written for distributed-memory architectures and includes performance results from a large Intel Xeon/Pentium 4 cluster.
More from Infra
- How infrastructure teams are using MCP-connected AI agents as copilot layers — BestRequirement7539 · 2026-07-26
- A OnePlus 12 can run Z Image Turbo and Flux.2 locally, but a 320×320 image still takes minutes — sgcego · 2026-07-26
- Founder Reveals Why They Stopped Fundraising: Compute Bottlenecks Outweigh Capital Needs — bookwormengr · 2026-07-26
- RTK is a Rust CLI proxy that cuts AI agent shell output by up to 90% — OpenAIDevs · 2026-07-26
- Inference providers may need cache policies that respect tool-call timeout settings — lucasmeijer · 2026-07-26
- OpenSmith adds local LLM tracing, live dashboards, and OpenTelemetry export — Maleficent-Emu-4549 · 2026-07-26