Dev disputes GPU non-determinism claims: Blackwell numerics are clear, defects just fail
mike64_t · x · 2026-10-07
Responding to claims that tensor cores have non-monotonic accumulation and that Blackwell's mesh NoC ordering perturbs numerics due to yield differences, mike64t pushes back:
- Non-monotonic accumulation is irrelevant to what determinism means; tcgen05 has a clear numerics model implementable in software with nothing "indeterministic"
- The yield-difference/NoC ordering claim is nearly unfalsifiable and something he has never observed
- He has seen blatant silicon defects in Blackwell (bits losing high states), but those produce wrong results (e.g., a result divided by two), not deterministically different ones—such chips should never pass silicon tests
- He argues the GPU non-determinism debate is overblown
Related event: Debating GPU Non-Determinism as NVIDIA CCCL 3.1 Adds Deterministic Controls(3 posts)→
More from Infra
- Google releases EmbeddingGemma 2: a 740M Apache 2.0 multimodal embedding model for fully local semantic search — lmoroney · 2026-10-07
- Musk: xAI will build and run its Terafab itself, TSMC may only sublease part of it — MickeySteamboat · 2026-10-07
- "72% of the intelligence with 3.8% of the GPUs": Mistral's compute-efficiency ratio sparks debate — cyb3rops · 2026-10-07
- Czech central bank governor installs 8 GPUs himself, gets told to buy more — misovalko · 2026-10-07
- Hugging Face hosts 568 plug-and-play compute kernels, filterable by 200+ hardware platforms — ariG23498 · 2026-10-07
- Intel CEO says Intel will stay in Musk's Terafab project after TSMC talk spooked investors — rohanpaul_ai · 2026-10-07