Custom sm_60 kernels push Tesla P100 decode from 11.6 to ~52 tok/s, repo open-sourced

Relative-Ingenuity15 · reddit · 2026-11-23

A Reddit user open-sourced a full set of custom Pascal (sm60) kernels, build flags and llama-swap configs for Tesla P100 16GB. Hand-written GEMVs, XMAD-based NVFP4 expert kernels, faster q80 attention and partial requantization lift Inkling-2-Mini 26B decode from 11.6 to 52 tok/s and prompt processing from 130 to 363 tok/s, with near-lossless quality (98% top-1 agreement, 0.0037 KL). Includes a Qwen3.8-Flash-Next IQ3S setup, per-kernel env-var toggles, and quant selection advice for different VRAM/RAM combos.

Original post →

More from Infra

Infra channel →