8-Hour Debug of ComfyUI Black Images Uncovers PyTorch FP16 Overflow Bug
rtsitola · reddit · 2026-08-01
A developer spent 8 hours debugging pure black image outputs and NaN errors in ComfyUI on a Quadro RTX 6000, uncovering three independent issues and providing a complete fix.
Core Issues & Fixes:
- PyTorch FP16 Overflow Bug: The main culprit. torch.mm() in FP16 overflows to infinity when input values exceed 44, affecting both Turing and Ampere GPUs. The fix is changing .half() to .bfloat16().
- Turing Startup Crash: ComfyUI may crash with cudaMallocAsync != native on Turing GPUs. Fix by setting the environment variable PYTORCHCUDAALLOCCONF=backend:cudaMallocAsync.
- VRAM Hardware Defect: If issues persist, the GPU might have physical VRAM damage (e.g., ECC errors).
The author open-sourced a GitHub repo with diagnostic scripts and launchers. The included NaN trap accurately pinpoints the exact ComfyUI module triggering the error.
More from Infra
- NXP Semiconductors in Talks to Acquire AI Chip Designer Ambarella — pstAsiatech · 2026-08-01
- Full 2.78T-parameter Kimi K3 Runs on Consumer Laptop via NVMe Streaming — rickasaurus · 2026-08-01
- CXMT's LPDDR6 Memory Nearing Mass Production with 12,800Mbps Speed — bookwormengr · 2026-08-01
- OpenAI Hits Git Perf Limits in Giant Monorepo, Upstreams Fixes — charliermarsh · 2026-08-01
- Why Chinese LLMs Struggle in AI Coding: The Hidden Costs of Compute and Quotas — 创业邦 · 2026-08-01
- Taalas Bakes Llama 3.1 into Custom Silicon, Hitting 15,000 Tokens/sec — generativist · 2026-08-01