Reddit: why I'm giving up on spec decoding in llama.cpp Vulkan — slower and memory-hungry

ParaboloidalCrest · reddit · 2026-11-19

A Reddit user details why speculative decoding in llama.cpp (Vulkan) feels broken: sometimes slower than plain decoding, no deterministic min/max parameter recipes, slower prompt processing, zero benefit at 128k+ context, and memory costs that could fund an extra 128k of context. Based on experience with Qwen 27B.

Original post →

More from Infra

Infra channel →