temperature=0 doesn't guarantee identical LLM outputs: floating-point and batching break determinism
cneuralnetwork · x · 2026-10-29
"temperature=0 means the same output every time" is a misconception. Greedy decoding still runs on nondeterministic inference: GPU floating-point ops aren't perfectly deterministic, batching and parallel execution perturb numerics, and near-tied token probabilities mean tiny differences flip the next token — after which the whole response can diverge. Deterministic decoding ≠ deterministic inference.
More from Models
- ChatGPT Pro/Business subscribers can now claim up to $500/month in API credits — JeremyNguyenPhD · 2026-10-29
- hunch: GPU-free ~20ms Jev-like decisions built on EmbeddingGemma 2 — iamrobotbear · 2026-10-29
- Rumor: DeepSeek's official API will strip cyber capabilities pending regulatory audits — teortaxesTex · 2026-10-29
- Vitalik: AI is hyper-knowledgeable but unwise, giving surface-level readings of new texts — paulnovosad · 2026-10-29
- No fresh local-capable open models lately — is Qwen 27B still the best? — emax · 2026-10-29
- Leaker says next model after Carbon is ready, waiting on OpenAI and Anthropic — iruletheworldmo · 2026-10-29