Why GPUs need philox, not xorshift: parallel RNG in AI training explained
abhi9u · x · 2026-09-25
abhi9u builds on a viral xorshift explainer to explain why traditional PRNGs don't work well on GPUs:
- CPU classic: George Marsaglia's xorshift (2003) is three lines of shifts and XORs, passes many standard randomness tests, and became the default RNG in several game engines.
- GPU differs: Sequential state-machine PRNGs are a poor fit for parallel hardware. AI training and inference need threads generating random numbers in parallel — while keeping the sequence reproducible given a seed, so threads can't just work independently.
- The answer: PyTorch and CUDA use the philox RNG, designed for fast parallel generation.
Bottom line: hardware shape dictates RNG design.
Related event: How PyTorch generates parallel random numbers on GPUs with Philox(2 posts)→
More from Infra
- Liquid AI ships LFM2.5-2.6B, a small fast model built for on-device agents on phones and laptops — helloiamleonie · 2026-09-25
- LFM2.5-2.6B matches Qwen3.5-9B at 3x size, unlocking on-device agents — helloiamleonie · 2026-09-25
- On-device agents need 3 things: ~4GB usable RAM, prefill speed, consistent quality — helloiamleonie · 2026-09-25
- Australia to host large share of next-gen datacenters, built for Anthropic — mattbeane · 2026-09-25
- Goldman: hyperscaler capex to grow 54% next year to $1.2 trillion — firstadopter · 2026-09-25
- Fine-tuning a 194M GLiNER2 dataset tagger on HF Jobs costs $1.50, lifting accuracy 10% to 69% — iamrobotbear · 2026-09-25