Qwen3.8-27B: Community Benchmarks and Speed Breakthroughs

Episode 1 · Developers Test Run 2.4T Parameter Qwen Model on Consumer Hardware (2026-08-14, 3 posts)

Several developers successfully ran the massively quantized 2.4T parameter Qwen model on high-end consumer hardware. Tests included a multi-GPU desktop setup achieving 0.8 tok/s and a 512GB Mac Ultra utilizing 1-bit quantization, proving the feasibility of running ultra-large models locally.

• Running 2.4T Qwen3.8 Model on RTX 5090 + 5060 Ti: 0.8 tok/s Tested — mossy_troll_84 · 2026-08-14
• Running Qwen 2.4T Locally: 5 GPUs Still Can't Make It Viable — klicker0 · 2026-08-14
• Running 2.4T Params on Mac Ultra: 1-bit Quantization Real-world Test — Ok_Technology_5962 · 2026-08-14

Episode 2 · Qwen3.8-27B Benchmarking on RTX 6000 (2026-08-15, 2 posts)

Reddit users initiated benchmarking for Qwen3.8-27B on RTX 6000 Pro, covering vLLM full precision, FP8, and BF16 TP2 configurations.

• Benchmark Request: Qwen3.8-27B Full Precision & FP8 on RTX 6000 Pro with vLLM — HumanDrone8721 · 2026-08-15
• Qwen3.8-27B benchmark: BF16 TP2 performance data — Maleficent_Bridge_41 · 2026-08-15

Episode 3 · Community Shares Qwen3.8-27B Deployment on 32GB VRAM (2026-08-15, 4 posts)

The community shared deployment configs for Qwen3.8-27B, including running on 32GB VRAM via llama.cpp and GGUF quantized models.

• Qwen3.8-27B Serving Configs: DGX Spark vLLM and RTX 4090 llama.cpp — erdaltoprak · 2026-08-15
• llama.cpp Config for Running Qwen3 27B on 32GB VRAM — ggerganov · 2026-08-15
• llama.cpp advanced usage: run Qwen3.8 27B on 32GB VRAM with speculative decoding — ggerganov · 2026-08-15
• ggml-org Releases Qwen3.8-27B-GGUF with Agent and Speculative Sampling — ggerganov · 2026-08-15

Episode 4 · RTX 3090 Optimized for Efficient Qwen3.8-27B Inference (2026-08-15, 3 posts)

Qwen3.8-27B runs efficiently on RTX 3090 at 34-36 t/s via llama.cpp and optimized runtimes like NInfer. Community efforts also produced INT8 quantized versions for dual 3090 setups.

• Optimized Dual 3090 Quantization of Qwen3.8-27B Released — luedtek · 2026-08-15
• RTX 3090 gets 35 t/s on Qwen 3.8 27B — cviperr33 · 2026-08-15
• NInfer Enables Efficient Qwen3.8-27B Inference on RTX 3090 — mrmontanasagrada · 2026-08-15

Episode 5 · Qwen3.8-27B Hits 50 tok/s on Dual RTX 3060s (2026-08-15, 2 posts)

A Reddit user demonstrated running Qwen3.8-27B on dual RTX 3060 12GB GPUs, achieving 50.6 tok/s using llama.cpp and MTP speculative decoding.

• Qwen3.8-27B hits ~40 tok/s on dual RTX 3060, passes CUDA coding test — anderspitman · 2026-08-15
• Running Qwen3.8-27B on dual RTX 3060s: 50 tok/s recipe — Ecstatic-Wash-7667 · 2026-08-15

Episode 6 · Qwen2.5 Local Performance Review (2026-08-15, 2 posts)

Evaluations show Qwen2.5 models run locally with high performance, rivaling Claude 3.5 Sonnet and Opus on consumer hardware.

• Qwen3.8 27B matches Opus 4.6 at home, running on MacBooks and RTX cards — daniel_mac8 · 2026-08-15
• Qwen2.5-72B local performance rivals Claude 3.5 Sonnet — AGI Hunt · 2026-08-15

Episode 7 · Qwen3.8-27B local benchmarks: from RTX 4080 Super to RTX 6000 Pro (2026-08-16, 7 posts)

Following the release of Qwen3.8-27B, the community ran extensive local benchmarks across GPUs from the RTX 3090 to the RTX 6000 Pro. High-end cards paired with SGLang and quantization can exceed 200 tok/s, while mainstream consumer GPUs (4090, 4080 Super) remain viable for long-context coding and everyday generation, indicating broad feasibility for local deployment.

Confirmed

• @iamMess reached 672 tps on an RTX 3090 via W4A16 quantization, FP8 KV cache, and int8 lmhead/embedtokens
• @StefanoGogioso relayed SGLang's Day-0 support: 206.1 tok/s decode on a single RTX 5090 with NVFP4 and DSpark speculative decoding; 38.28 tok/s on DGX Spark
• @BanghuaZ relayed: RTX 6000 Pro (96GB) achieves 200-223 tok/s single-stream with SGLang, NVFP4 weights and DSpark, supporting native 256k context
• @rohanpaulai shared a demo at 115 tok/s on RTX 5090 (32GB VRAM); official BF16 weights are 55.6 GB
• @julianharris measured 73 tok/s on RTX 4090 with unsloth 4bit at 128k context for coding; disabling thinking mode raises it to 78 tok/s with notably lower quality
• @BopSupreme reported 39 tok/s on RTX 4080 Super (16GB), TTFT 200-230ms, 34 tok/s under a 26K-token real workload
• @cocktailpeanut relayed user zast57: RTX 4090 via Pinokio one-click Ollama + OpenWebUI runs the 27B model near 100 tok/s on proofreading tasks

Unconfirmed

• @iamMess's 672 tps figure does not clarify prefill vs. decode, a large gap versus other measurements

Why it matters

• Extreme optimization cases and high-end results show the hardware ceiling, while smooth mid-range performance proves broad local deployability
• The thinking-mode speed/quality trade-off and the 115 vs. 206 tok/s comparison on 5090 offer concrete guidance for deployment choices

• Qwen 27B hits nearly 100 tokens/s on a local RTX 4090 via Ollama and Pinokio — cocktailpeanut · 2026-08-16
• Qwen3.8-27B hits 223 tok/s on RTX 6000 Pro with NVFP4 — BanghuaZ · 2026-08-17
• Qwen 3.8 27b Local Coding Test: 73 tok/s at 128k Context, Usable Performance — julianharris · 2026-08-17
• RTX 3090 runs Qwen3.8-27B at 672 tps via extreme quantization — iamMess · 2026-08-17
• Qwen3.8-27B hits 206 tok/s on single RTX 5090 via SGLang — StefanoGogioso · 2026-08-17
• Qwen3.8 27B Hits 39 tok/s on RTX 4080 Super, Good for Local Coding — BopSupreme · 2026-08-17
• RTX 5090 Demo: Running Qwen 3.8 27B Locally at 115 tok/s — rohanpaul_ai · 2026-08-17

Episode 8 · Running Local Agent Models on Mac: Memory Is the Deciding Factor (2026-08-16, 2 posts)

Mac users are testing local agent models, from a 16GB M4 Mac mini to a 64GB Mac Studio running community MLX 4-bit quantized Qwen 2.5, with unified memory size emerging as the key factor in model choice and performance.

• Running Local Agent Models on M4 Mac mini with 16GB: What Actually Works? — cuberhino · 2026-08-16
• Running Qwen 2.5 on Mac Studio: Performance Tests and Agent Struggles — Over_Technology_1764 · 2026-08-16

Episode 9 · RTX 3090 users seek best local setup for running Qwen3 models (2026-08-16, 2 posts)

RTX 3090 owners on Reddit are asking how to best run Qwen3 models locally, discussing OS choices, recommended software, and inference engines such as vLLM and llama.cpp to optimize performance within limited VRAM.

• Optimizing Qwen3.8 config for single RTX 3090 setups — sagiroth · 2026-08-16
• How to start running Qwen 3.8 locally with a 3090 GPU? — ozymandizz · 2026-08-17

Episode 10 · Qwen3.8-27B local deployment tested: stable single-GPU agent runs and 128K context (2026-08-16, 3 posts)

Developers tested Qwen3.8-27B locally: stable 10-hour agent runs on an RTX 3090 with auto context compression, five-hour sessions on a 95GB Blackwell card, and 73 tok/s at 128K context with 4-bit quantization on an RTX 4090.

• Qwen 3.8 27b with DeepSeek Harness: 10h stable, 37-60 tok/s on RTX 3090 — cviperr33 · 2026-08-16
• Qwen 3.8 27B Local Coding: 73 tok/s at 128K Context on RTX 4090 — julianharris · 2026-08-17
• 5-Hour Local Coding Agent Session on Qwen3.8-27B: 97 tok/s Streaming, 70% Context Compaction — illgettheownerforyou · 2026-08-18

Episode 11 · Qwen3.8-27B Inference Optimized for RTX 3090 (2026-08-18, 2 posts)

Developers released highly optimized inference for Qwen3.8-27B on a single RTX 3090, leveraging fp16 recurrent states, full-layer int8 activations, and draft vocabulary tricks to reach up to 124 TPS with greedy sampling.

• Qwen3.8-27B optimization hits 1150 tps on RTX 3090 — iamMess · 2026-08-18
• Pushing Qwen3.8-27B to 124 tps on a single RTX 3090 — iamMess · 2026-08-19

Episode 12 · Inco AI Launches DFlash 2, Speeding Up LLM Inference Up to 4.6x (2026-08-19, 4 posts)

Inco AI released DFlash 2, an inference acceleration technique delivering up to 4.6x speedup over standard autoregressive decoding, running Qwen3.8-27B at 70 tok/s on an M5 Max MacBook Pro.

• DFlash 2: Qwen3.8-27B hits 70 tok/s on Apple M5 Max — Lianhuiq · 2026-08-19
• DFlash 2: Qwen3.8-27B hits 70 tok/s on MacBook with 4.6x speedup — songhan_mit · 2026-08-19
• DFlash 2 released: up to 4.6× speedup for AI inference — igilitschenski · 2026-08-19
• DFlash 2 runs a 27B model at 70 tok/s on M5 Max, up to 4.6x faster — DevDminGod · 2026-08-19

Episode 13 · llama.cpp's New DFlash2 Boosts Qwen Inference Speed Up to 3x (2026-08-20, 2 posts)

Community tests show llama.cpp's new DFlash2 feature speeds up Qwen 3.8 27B inference by up to 3x on RTX 6000, though one RTX 5090 test found it 20% faster than MTP at the cost of a 38% smaller context.

• llama.cpp dflash2: Qwen 3.8 27B Inference Speed Up to 3x — Top-Eye-8104 · 2026-08-20
• llama.cpp Benchmark: DFlash2 20% Faster but Cuts Context by 38% — Opening-Broccoli9190 · 2026-08-21

Episode 14 · Qwen3.8-27B Inference Hits 381 TPS on RTX 3090 (2026-08-20, 2 posts)

A developer optimized Qwen3.8-27B inference on an RTX 3090 to reach 134 TPS with multi-turn response latency cut from 23s to 1s, then pushed reproduction speed to 381 TPS using longer verification blocks.

• Qwen3.8-27B hits 134 TPS on RTX 3090 with DFlash2 and custom optimizations — iamMess · 2026-08-20
• Pushing Qwen3.8-27B to 381 tps on a single RTX 3090 — iamMess · 2026-08-21