8-Hour Debug of ComfyUI Black Images Uncovers PyTorch FP16 Overflow Bug
rtsitola · reddit · 2026-08-01
A developer spent 8 hours debugging pure black image outputs and NaN errors in ComfyUI on a Quadro RTX 6000, uncovering three independent issues and providing a complete fix.
Core Issues & Fixes:
- PyTorch FP16 Overflow Bug: The main culprit. torch.mm() in FP16 overflows to infinity when input values exceed 44, affecting both Turing and Ampere GPUs. The fix is changing .half() to .bfloat16().
- Turing Startup Crash: ComfyUI may crash with cudaMallocAsync != native on Turing GPUs. Fix by setting the environment variable PYTORCHCUDAALLOCCONF=backend:cudaMallocAsync.
- VRAM Hardware Defect: If issues persist, the GPU might have physical VRAM damage (e.g., ECC errors).
The author open-sourced a GitHub repo with diagnostic scripts and launchers. The included NaN trap accurately pinpoints the exact ComfyUI module triggering the error.
More from Infra
- GitHub rewrote the 800k-line Copilot runtime in Rust using AI agents, cutting CPU and memory by over 90% — bibryam · 2026-10-16
- NVIDIA quietly delists the entry-level RTX 5050 from its marketplace, following the RTX 5090 — chemist_slime · 2026-10-16
- Grok claims 816B MoE model runs locally on dual M5 Ultra Mac Studios via MLX — grok · 2026-10-16
- Same 1,000 Cloudflare Durable Objects: $4,050/month cut to $22 with one trick — threepointone · 2026-10-16
- NVIDIA ships FP4-quantized Qwen3.8-Flash for local use, trending on Hugging Face — nvidia · 2026-10-16
- Baidu's Virtual Head Attention tames the KV cache, FLOPs, quality triangle in long-context LLMs — baidu · 2026-10-16