Testing Qwen3.8-27B with 131k Context on 16GB VRAM using Speculative Decoding
BuffMcBigHuge · reddit · 2026-08-20
The author tested Qwen3.8-27B on an RTX 4080 16GB to maximize context and performance. By using a custom llama.cpp build with the DFlash2 branch, ngram-mod sampling, and MTP (Medusa Thought Proposal) speculative decoding, they achieved stable context lengths up to 131k. The post compares results across different configurations like IQ3XXS quantization and provides full startup commands.
More from Infra
- Haight Street Compute Market: a one-day trading floor for GPU deals in SF — ctjlewis · 2026-10-16
- Temporal Serverless Workers on Cloud Run: queue-driven autoscaling, scale to zero — rseroter · 2026-10-16
- LMA brings day-0 model support to TPU for the first time, spanning NVIDIA, AMD, Intel via SGLang — ying11231 · 2026-10-16
- Semtech's analyst-day slide cleanly explains the four signal-integrity impairments — BenBajarin · 2026-10-16
- Fortytwo Engine boosts Qwen3.8-27B accuracy 7% and cuts tokens 25% via Swarm Inference — rohanpaul_ai · 2026-10-16
- Berkeley scholars: concentrated chip supply chain makes a hardwired frontier AI pause feasible — wfithian · 2026-10-16