Benchmark sweep shows greedy decoding tanks coding accuracy: 70% at temp 0 vs 82.7% at temp 1.0
Brief-Tap-6616 · reddit · 2026-10-30
A Reddit user systematically swept temperature on LiveCodeBench v6 (100 hardest problems) with fixed samplers (topp 0.95, topk 20, minp 0), running Qwen3.8 IQ4XS-M on llamAmpere with medium reasoning effort:
- Coding accuracy rises with temperature: 70.0% at temp 0, 76.0% at 0.25, 76.0% at 0.5, 79.7% at 0.75, 82.7% at 1.0; gaps at 0 and 0.25 are statistically significant
- Loop rate: 9% at temp 0 and 6% at 0.25, vs just 0.3% at temp 1.0
- Low temps also make answers more verbose; GPQA Diamond (recall) scores were unaffected but answers ran 8% longer
The author notes many speculative-decoding benchmarks use greedy decoding because it's easiest to accelerate — but it removes the token randomness behind non-recall performance. Temp-0 scores roughly match quants 40% smaller. Follow-up tests at 0.8/0.9 are underway, judged by wall-clock time. Dataset released on HuggingFace.
More from Models
- Researcher Finds 600+ Bugs in LLM Kernel Code; Claude Blocks Safety Analysis — pmddomingos · 2026-10-31
- Signal65: Snowflake's CoCo agent finishes workflow 4.6x faster than manual — ryanshrout · 2026-10-31
- Mistral Large 4 matches GLM on coding with 47% fewer tokens, halves preview price — petrusenko_max · 2026-10-31
- Intelligence per dollar fell ~1,000,000x in three years, shows double-exponential chart — banteg · 2026-10-31
- Altman: each math AI proof is worth $10M, and cost will drop 100x within a year — Traditional-Chip8339 · 2026-10-31
- GPT high-tier model accused of task-abandoning and sycophancy in coding and lyrics — Kyrannio · 2026-10-31