FULL STORY

Qwen3.8-27B: A Community Benchmark Frenzy

After Qwen3.8-27B launched, the community raced to benchmark it locally on consumer GPUs and Macs, pushing speeds up to 672 tps with quantization and speculative decoding tricks.

2026-08-14 ~ 2026-08-17 · 10 episodes · 28 posts

Episode 1 · Developers Test Run 2.4T Parameter Qwen Model on Consumer Hardware (2026-08-14, 3 posts)

Several developers successfully ran the massively quantized 2.4T parameter Qwen model on high-end consumer hardware. Tests included a multi-GPU desktop setup achieving 0.8 tok/s and a 512GB Mac Ultra utilizing 1-bit quantization, proving the feasibility of running ultra-large models locally.

Running 2.4T Qwen3.8 Model on RTX 5090 + 5060 Ti: 0.8 tok/s Tested — mossy_troll_84 · 2026-08-14
Running Qwen 2.4T Locally: 5 GPUs Still Can't Make It Viable — klicker0 · 2026-08-14
Running 2.4T Params on Mac Ultra: 1-bit Quantization Real-world Test — Ok_Technology_5962 · 2026-08-14

Episode 2 · Qwen3.8-27B Benchmarking on RTX 6000 (2026-08-15, 2 posts)

Reddit users initiated benchmarking for Qwen3.8-27B on RTX 6000 Pro, covering vLLM full precision, FP8, and BF16 TP2 configurations.

Benchmark Request: Qwen3.8-27B Full Precision & FP8 on RTX 6000 Pro with vLLM — HumanDrone8721 · 2026-08-15
Qwen3.8-27B benchmark: BF16 TP2 performance data — Maleficent_Bridge_41 · 2026-08-15

Episode 3 · Community Shares Qwen3.8-27B Deployment on 32GB VRAM (2026-08-15, 4 posts)

The community shared deployment configs for Qwen3.8-27B, including running on 32GB VRAM via llama.cpp and GGUF quantized models.

Qwen3.8-27B Serving Configs: DGX Spark vLLM and RTX 4090 llama.cpp — erdaltoprak · 2026-08-15
llama.cpp Config for Running Qwen3 27B on 32GB VRAM — ggerganov · 2026-08-15
llama.cpp advanced usage: run Qwen3.8 27B on 32GB VRAM with speculative decoding — ggerganov · 2026-08-15
ggml-org Releases Qwen3.8-27B-GGUF with Agent and Speculative Sampling — ggerganov · 2026-08-15

Episode 4 · RTX 3090 Optimized for Efficient Qwen3.8-27B Inference (2026-08-15, 3 posts)

Qwen3.8-27B runs efficiently on RTX 3090 at 34-36 t/s via llama.cpp and optimized runtimes like NInfer. Community efforts also produced INT8 quantized versions for dual 3090 setups.

Optimized Dual 3090 Quantization of Qwen3.8-27B Released — luedtek · 2026-08-15
RTX 3090 gets 35 t/s on Qwen 3.8 27B — cviperr33 · 2026-08-15
NInfer Enables Efficient Qwen3.8-27B Inference on RTX 3090 — mrmontanasagrada · 2026-08-15

Episode 5 · Qwen3.8-27B Hits 50 tok/s on Dual RTX 3060s (2026-08-15, 2 posts)

A Reddit user demonstrated running Qwen3.8-27B on dual RTX 3060 12GB GPUs, achieving 50.6 tok/s using llama.cpp and MTP speculative decoding.

Qwen3.8-27B hits ~40 tok/s on dual RTX 3060, passes CUDA coding test — anderspitman · 2026-08-15
Running Qwen3.8-27B on dual RTX 3060s: 50 tok/s recipe — Ecstatic-Wash-7667 · 2026-08-15

Episode 6 · Qwen2.5 Local Performance Review (2026-08-15, 2 posts)

Evaluations show Qwen2.5 models run locally with high performance, rivaling Claude 3.5 Sonnet and Opus on consumer hardware.

Qwen3.8 27B matches Opus 4.6 at home, running on MacBooks and RTX cards — daniel_mac8 · 2026-08-15
Qwen2.5-72B local performance rivals Claude 3.5 Sonnet — AGI Hunt · 2026-08-15

Episode 7 · Qwen3.8-27B Benchmarks: 672 tps on RTX 3090 (2026-08-16, 6 posts)

Local benchmarks of Qwen3.8-27B have poured in since its release: SGLang offers Day-0 support, and with NVFP4 quantization plus DSpark speculative decoding, a single RTX 5090 decodes at 206.1 tok/s while the RTX 6000 Pro (96GB) hits 200-223 tok/s in single-stream throughput; the RTX 4090 delivers 73 tok/s for coding with a 128k context, which testers deem fully usable. A 27B-class model now runs smoothly on a single consumer GPU.

Confirmed

Relayed by @StefanoGogioso: SGLang provides Day-0 support for Qwen3.8-27B; an RTX 5090 with NVFP4 and DSpark decodes at 206.1 tok/s, and 38.28 tok/s on DGX Spark
Relayed by @BanghuaZ: an SGLang script on the RTX 6000 Pro (96GB) using NVFP4 weights plus DSpark speculative decoding reaches 200-223 tok/s single-stream throughput with native long-context support (post summary truncated; exact figures not fully shown)
@iamMess pushed inference to 672 tps on an RTX 3090 through extreme optimizations: W4A16 quantization, FP8 KV Cache, and converting lmhead and embedtokens to int8
@cocktailpeanut relayed user zast57: with Pinokio one-click deployment of Ollama + OpenWebUI, an RTX 4090 runs the Qwen 3 series 27B at nearly 100 token/s for proofreading and other text tasks
@julianharris tested local coding on an RTX 4090 with unsloth 4-bit quantization: 73 tok/s at 128k context, fully usable; disabling thinking mode speeds things up to 78 tok/s but quality drops noticeably

Unconfirmed

@iamMess's 672 tps does not specify whether it is decode or prefill speed; it sits far above other cards' decode rates ranging from the tens to the low 200s, and the measurement methodology awaits clarification from the author
Model names vary across posts (e.g., Qwen3.6-28B, Qwen 3.28:27b); this roundup standardizes on Qwen3.8-27B

Why it matters

Locally runnable at launch: SGLang Day-0 support, NVFP4, speculative decoding, unsloth quantization, and Pinokio one-click deployment landed almost simultaneously, dramatically shortening the cycle from new model release to consumer-GPU usability
128k long-context coding still hits 73 tok/s on a single 4090, so mid-range GPUs can already shoulder everyday coding-assistant workloads
The thinking-mode trade-off (a small speedup when disabled but noticeably worse quality) gives local-deployment users a direct configuration reference

Qwen 27B hits nearly 100 tokens/s on a local RTX 4090 via Ollama and Pinokio — cocktailpeanut · 2026-08-16
Qwen3.8-27B hits 223 tok/s on RTX 6000 Pro with NVFP4 — BanghuaZ · 2026-08-17
Qwen 3.8 27b Local Coding Test: 73 tok/s at 128k Context, Usable Performance — julianharris · 2026-08-17
RTX 3090 runs Qwen3.8-27B at 672 tps via extreme quantization — iamMess · 2026-08-17
Qwen3.8-27B hits 206 tok/s on single RTX 5090 via SGLang — StefanoGogioso · 2026-08-17
Qwen3.8 27B Hits 39 tok/s on RTX 4080 Super, Good for Local Coding — BopSupreme · 2026-08-17

Episode 8 · Running Local Agent Models on Mac: Memory Is the Deciding Factor (2026-08-16, 2 posts)

Mac users are testing local agent models, from a 16GB M4 Mac mini to a 64GB Mac Studio running community MLX 4-bit quantized Qwen 2.5, with unified memory size emerging as the key factor in model choice and performance.

Running Local Agent Models on M4 Mac mini with 16GB: What Actually Works? — cuberhino · 2026-08-16
Running Qwen 2.5 on Mac Studio: Performance Tests and Agent Struggles — Over_Technology_1764 · 2026-08-16

Episode 9 · RTX 3090 users seek best local setup for running Qwen3 models (2026-08-16, 2 posts)

RTX 3090 owners on Reddit are asking how to best run Qwen3 models locally, discussing OS choices, recommended software, and inference engines such as vLLM and llama.cpp to optimize performance within limited VRAM.

Optimizing Qwen3.8 config for single RTX 3090 setups — sagiroth · 2026-08-16
How to start running Qwen 3.8 locally with a 3090 GPU? — ozymandizz · 2026-08-17

Episode 10 · Qwen 3.8 27B Shows Stable Local Performance (2026-08-16, 2 posts)

Developers tested Qwen 3.8 27B on RTX 3090 and 4090 GPUs, showing stable performance at 37-73 tok/s with 4bit quantization and 128K context support.

Qwen 3.8 27b with DeepSeek Harness: 10h stable, 37-60 tok/s on RTX 3090 — cviperr33 · 2026-08-16
Qwen 3.8 27B Local Coding: 73 tok/s at 128K Context on RTX 4090 — julianharris · 2026-08-17