Softmax temperature can make a wrong AI prediction look 99% certain — without changing a weight
techNmak · x · 2026-11-11
A clear explainer on why reported confidence and actual accuracy are separate properties. With logits (2, 1, 0), setting softmax temperature T=2 gives the top class 50.65% probability; T=0.2 pushes it to 99.33% — with no change to inputs, weights, or the predicted class.
- Temperature scaling preserves logit ordering, so top-1 accuracy stays identical; confidence numbers alone say nothing about correctness
- A 2017 ICML paper showed temperature scaling fitted on held-out data substantially improves probability calibration
- The same math governs LLM sampling: lower temperature concentrates on high-scoring tokens, higher temperature diversifies outputs — parameters unchanged
Adjusting a probability distribution adds no knowledge; whether 99% confidence is reliable must be established through evaluation.
More from Models
- Developer Says He Stopped Testing Model Configs After Carbon Landed — cgarciae88 · 2026-11-11
- Google's AudioFM Foundation Model Tops Audio Benchmarks, Powers 2B Daily Predictions — umpedronosapato · 2026-11-11
- Unverified Rumor: GPT 6.2 Astra and Possibly 6.2 Sol Landing This Thursday — imjustnewatai · 2026-11-11
- Pelican SVG test: Fable 5.2 takes 25 min/$25 vs Gemini 4 Hydrogen's 5 minutes — FinanceYF5 · 2026-11-11
- Same task, 4 AI workflows: Fable 5.2 wins at $6/2min vs GPT-6 Astra's $93/23min — FinanceYF5 · 2026-11-11
- Critic: Opus 5.5 vs Fable 5.2 demos lack prompts, cost and timing data — FinanceYF5 · 2026-11-11