ETH SPCL paper on performance foundations of parallel reasoning LLM training wins best poster award
thoefler · x · 2026-10-29
ETH Zurich's SPCL group won Best Poster Presentation at the 3rd EFCL International Workshop on Sustainable AI for "Performance Foundations of Parallel & Distributed Reasoning Language Models" (arXiv:2608.27046).
Authors include Maciej Besta, Leonard Schmidt and Torsten Hoefler. The paper tackles the massive compute footprint of training reasoning LLMs like DeepSeek-R1, o3 and Kimi k1.5, which require millions of GPU-hours and tightly coupled multi-model pipelines. It systematizes the RL-for-LLM paradigm with a compute-centric analysis of PPO, GRPO and variants, and builds a taxonomy of intra- and inter-model parallelism strategies (data, tensor, pipeline, sequence, context, expert parallelism) to enable high-performance, scalable and cost-effective RLM training.
More from Infra
- PyTorch 2.15 ships with default NCCL rewrite, NVFP4 decode on Blackwell — PyTorch · 2026-10-29
- US data center construction hits $85B annual pace, up 73% year over year — BenBajarin · 2026-10-29
- Azure Cosmos DB recipe: isolated disposable databases for a swarm of parallel coding agents — adnan_hashmi · 2026-10-29
- Short seller claims Cerebras wafers degrade after ~2 years, undermining its business — himanshustwts · 2026-10-29
- Apple's RAM upgrade pricing: 512GB costs a staggering $13,950 over base — firstadopter · 2026-10-29
- AI data center gas buildout could cost over $5.6 billion a year in health impacts — AINowInstitute · 2026-10-29