llama.cpp Precision Bug Fixed on P100

apollo_mg · reddit · 2026-07-12

The author discovered that llama.cpp on **sm_60 P100** GPUs incorrectly takes an "fp16 is faster" CUDA path, causing precision loss. While **GTX 10 series / P40** were already excluded, P100 fell victim due to a missed check. The provided fix is just **3 lines** long. Using **Qwen3.6-27B** and **wikitext-2** to compare against fp32 reference logits: - **Median KLD** dropped from **0.0023 to 0.000001**, a roughly **2300x** improvement - **Top-token agreement** increased from **96.5% to 99.9%** Performance-wise, this "fix" comes at no cost: across tests on 27B hybrid, 4B dense, and 36B MoE models at 8k context, prefill speed showed no difference, and decode was actually about **1.4% faster**. The author stressed that this only affects **sm_60** and won't impact 4090/Volta or newer architectures. The post concludes by noting that the patch has already been merged/queued in relevant forks, with full methods and data available in their technical documentation.

Original post →

More from Infra

Infra channel →