llama.cpp Precision Bug Fixed on P100
apollo_mg · reddit · 2026-07-12
The author discovered that llama.cpp on **sm_60 P100** GPUs incorrectly takes an "fp16 is faster" CUDA path, causing precision loss. While **GTX 10 series / P40** were already excluded, P100 fell victim due to a missed check. The provided fix is just **3 lines** long. Using **Qwen3.6-27B** and **wikitext-2** to compare against fp32 reference logits: - **Median KLD** dropped from **0.0023 to 0.000001**, a roughly **2300x** improvement - **Top-token agreement** increased from **96.5% to 99.9%** Performance-wise, this "fix" comes at no cost: across tests on 27B hybrid, 4B dense, and 36B MoE models at 8k context, prefill speed showed no difference, and decode was actually about **1.4% faster**. The author stressed that this only affects **sm_60** and won't impact 4090/Volta or newer architectures. The post concludes by noting that the patch has already been merged/queued in relevant forks, with full methods and data available in their technical documentation.
More from Infra
- Fluidstack raises $830M at $7.5B valuation as Anthropic backs a $50B compute buildout — rohanpaul_ai · 2026-07-21
- Early Krea2 Gradio WebUI targets 6GB low-VRAM local runs — Fluid_Kaleidoscope17 · 2026-07-21
- Z.AI starts running a 1GW AI data center built entirely on domestic chips — Polymarket · 2026-07-21
- Local models feel far more capable once paired with the right harness — Soft-Barracuda8655 · 2026-07-21
- Voice-agent teams should use platforms first, then own STT events when failures get weird — FollowingSuitable941 · 2026-07-21
- Alibaba open-sources T-Head SAIL, the software stack behind its AI chips — 量子位 · 2026-07-21