gfx906-llama-cpp Fork Boosts MI50/MI60 Inference +23% Prefill, Fits 250k Context on 40GB
milpster · reddit · 2026-09-05
Developer milpster shipped a major update to gfx906-llama-cpp, a fork optimizing llama.cpp for AMD GCN cards like MI50/MI60/Radeon VII: prefill PP16384 up from 332.5 to 410 t/s (+23%), 120k deep fill +14%, TG decode +11% (15.1 t/s), and 250k context squeezed into 40GB VRAM (upstream can't fit it). Outputs are bit-identical to upstream. Gains came mostly from adopting relevant upstream PRs; the author notes the fork was built with AI assistance. A practical resource for cheap local LLM inference on aging GPUs.
More from Infra
- llama.cpp team launches Llama: one-click local AI on your Mac menu bar — Hugging Face · 2026-10-23
- China's daily token use hits 500T, up 4x as average price doubles to ¥13.46 — teortaxesTex · 2026-10-23
- New all-reduce kernels deliver 12% decode speedup for DeepSeek-V4.1 on PCIe-connected GPUs — AlpinDale · 2026-10-23
- Offshore oil rigs could be converted into AI data centers powered by stranded gas — clemnt · 2026-10-23
- Same native UI, two stacks: Swift build is ~80MB Mac-only, Zig build is ~10MB cross-platform — jedisct1 · 2026-10-23
- SurfSense: an open-source local NotebookLM with MCP agents and a prompt-caching lesson — FurtiveMirth · 2026-10-23