torch.compile skips guards for FSDP, reducing MFU in dynamic shape training
SeunghyunSEO7 · x · 2026-08-24
User reports that torch.compile automatically skips guard installation for FSDP-wrapped modules (behavior persisting in FSDP2). In dynamic shape=True setups like Flux.2 Klein multi-resolution training, this generates excessive symbolic arguments, hindering fusion of element-wise operations like layernorm and rope, and significantly reducing MFU.
Related event: torch.compile with FSDP Dynamic Shapes Degrades MFU(3 posts)→
More from Infra
- Data centers use 0.04% as much water as farms, debunking scarcity myths — davidpattersonx · 2026-08-24
- Spooqy Roadmap: Automating Trustless Software Verification with AI Agents — StefanoGogioso · 2026-08-24
- Open-weight token share doubled in a year; agents now use 14x human tokens — Exponential View (Azeem Azhar) · 2026-08-24
- Faster Alternatives to llama.cpp for Local LLM Inference? — PoshoZen11 · 2026-08-24
- Navigating the Jungle of MLX Distros for Local LLMs on Mac — mouseofcatofschrodi · 2026-08-24
- OpenAI contributes Git upstream optimizations for large monorepos — charliermarsh · 2026-08-24