Debating GPU Non-Determinism as NVIDIA CCCL 3.1 Adds Deterministic Controls

Developers pushed back on claims that GPUs make LLM outputs inherently nondeterministic, while NVIDIA's CCCL 3.1 introduced three tiers of floating-point determinism control, with cross-GPU consistency costing 20-30% performance.

2026-10-06 ~ 2026-10-07 · 3 related posts

• blelbach: GPU parallelism isn't a random number generator, non-determinism is an implementation gap — blelbach · 2026-10-06
• NVIDIA CCCL 3.1 adds three-tier FP determinism control; cross-GPU reproducibility costs 20-30% — blelbach · 2026-10-07
• Dev disputes GPU non-determinism claims: Blackwell numerics are clear, defects just fail — mike64_t · 2026-10-07