Custom Qwen3.8-27B engine doubles llama.cpp speed on a 3090 with exact greedy outputs

Nice_Profession_9078 · reddit · 2026-11-02

A Reddit user published detailed benchmarks of their Qwen3.8-27B engine (qwen38-megakernel) against llama.cpp. Key finding: speculative decoding is supposed to be lossless, but llama.cpp changed its greedy answer on 9 of 282 benchmark questions just by going from 4 to 7 MTP drafts (8 reproduce reliably), while this engine produces identical token IDs at 0, 4 or 7 drafts.

Results on the same RTX 3090, same unsloth UD-Q4KXL model file, one session:

Greedy short prompts: 1.5-2.1x (code writing 232.9 vs 110.8 tok/s; migration plan 158.7 vs 89.8)
Sampled with thinking: 138 vs 73 tok/s (1.9x)
100K context: 109 vs 54 tok/s (2.0x); the full 262K context fits in 24 GB, decoding at 66.6 tok/s at 261K depth (llama.cpp q80 cache tops out 160K)
With drafting off on both sides, still 7.5% faster on short prompts and 41% at 100K—faster attention kernels too
Four concurrent agents: 382 vs 137 tok/s (2.8x)
Editing an earlier message in an 11.7K chat: 76ms vs 266ms (3.5x)

Key implementation: the whole cycle (main pass, lmhead, verify, MTP catch-up, next drafts) runs as one persistent kernel on int8 tensor cores—verifying 8 tokens costs 29 ms vs 22 ms for one. Full numbers and methodology in the repo's BENCHMARKS.md (github: cutlerbenjamin1-cmd/qwen38-megakernel).

Original post →

Custom Qwen3.8-27B engine doubles llama.cpp speed on a 3090 with exact greedy outputs

More from Infra