Why temperature=0 still can't guarantee deterministic LLM inference
A blog post from Thinking Machines Lab explains why LLM inference remains nondeterministic even with temperature=0: factors like batch shapes and kernel reduction order still affect logits beyond the sampling step.
2026-10-07 ~ 2026-10-07 · 2 related posts
- Temp 0 doesn't guarantee deterministic LLM output — batch shape and kernels shift logits — JFPuget · 2026-10-07
- Thinking Machines explains why LLM inference stays nondeterministic even at temperature 0 — JFPuget · 2026-10-07