blelbach: GPU parallelism isn't a random number generator, non-determinism is an implementation gap
blelbach · x · 2026-10-06
blelbach pushes back hard on the claim that massive GPU parallelism makes LLM outputs inherently random: "GPU parallelism is not a random number generator." He acknowledges that non-determinism exists in some implementations — thread sync order affecting floating-point rounding — but argues it's an implementation gap, not a fundamental design flaw.
More from Research
- Bayesian teaching dramatically improves probabilistic reasoning in LLMs, Nature Comms paper finds — tallinzen · 2026-10-06
- LiFT loops a DiT at inference: beats DiT-XL/2 with 52% less inference compute — cgmsnoek · 2026-10-06
- Researcher laments how slow humans look as algorithm design breakthroughs pile up — michaelchchoi · 2026-10-06
- Meta paper: dual coding agents cross-review lift correct patches from 45.8% to 62.5% — rohanpaul_ai · 2026-10-06
- Study of 227K STEM PhD abstracts links AI use to leaving academia for industry — ValerioCapraro · 2026-10-06
- Subliminal Steering presented at COLM: a more consistent take on subliminal learning — johnhewtt · 2026-10-06