Dual RTX 6000s run Qwen3.8-27B at 150t/s, yet 12x slower than Claude on the task
EkbatDeSabat · reddit · 2026-08-24
The poster runs Qwen3.8-27B FP8 on two RTX 6000 Pro cards via SGLang at 150 tokens/sec—yet a documentation-heavy baseline task takes nearly 2 hours vs under 10 minutes on Opus 5, a 12x gap. The task eats 80k context before starting; the primary agent spawns sub-agents, one offloading to the second GPU.
Full deployment config is shared: one SGLang Docker container per GPU with 256k context, FP8 KV cache (fp8e4m3), mem-fraction-static 0.90, max-running-requests 4, chunked prefill 8192, qwen3 reasoning/tool-call parsers, and EAGLE speculative decoding (3 steps, topk 1, 4 draft tokens), among other flags.
Findings so far: BF16 primary + FP8 offload is slower (3 hours); 500k context, though unsupported, beats 256k by an hour. Claude Code, pi code, qwen code, and Hermes all behave similarly. The author wonders if this is the inherent cost of the 27B's thinking, and plans to test alternatives to SGLang next.
More from coding & agent
- Repo includes Claude and Codex implementations — tekbog · 2026-08-25
- Developer Argues Codex Remains the Best AI Coding Product — nickbaumann_ · 2026-08-25
- Apodex 1.1 mini: open 35B local model, harness swap adds up to 10 points — SimonShaoleiDu · 2026-08-25
- How AI Agents Understand Design System Languages — round · 2026-08-25
- From Scripts to Loops: Engineering Challenges and Solutions for Production Agentic Systems — Pavan_Belagatti · 2026-08-25
- AWS launches Agent Registry and backs ARD, an open spec for cross-environment agent discovery — AWS ML Blog · 2026-08-25