Dual RTX 6000s run Qwen3.8-27B at 150t/s, yet 12x slower than Claude on the task
EkbatDeSabat · reddit · 2026-08-24
The poster runs Qwen3.8-27B FP8 on two RTX 6000 Pro cards via SGLang at 150 tokens/sec—yet a documentation-heavy baseline task takes nearly 2 hours vs under 10 minutes on Opus 5, a 12x gap. The task eats 80k context before starting; the primary agent spawns sub-agents, one offloading to the second GPU.
Full deployment config is shared: one SGLang Docker container per GPU with 256k context, FP8 KV cache (fp8e4m3), mem-fraction-static 0.90, max-running-requests 4, chunked prefill 8192, qwen3 reasoning/tool-call parsers, and EAGLE speculative decoding (3 steps, topk 1, 4 draft tokens), among other flags.
Findings so far: BF16 primary + FP8 offload is slower (3 hours); 500k context, though unsupported, beats 256k by an hour. Claude Code, pi code, qwen code, and Hermes all behave similarly. The author wonders if this is the inherent cost of the 27B's thinking, and plans to test alternatives to SGLang next.
More from coding & agent
- TRL adds day-0 support for fine-tuning Muse Spark Open's 816B MoE with LoRA on 64 H100s — QGallouedec · 2026-10-16
- Databricks engineers land impactful PRs on day 2 with AI dev tools, vs week 3 before — Yuchenj_UW · 2026-10-16
- Perplexity open-sources SPACE, the sandbox platform powering its Computer agents — AravSrinivas · 2026-10-16
- Vercel makes Python generally available: FastAPI, Django, Fluid Compute support — steren · 2026-10-16
- Full course: build a playable Unity fishing game with Claude Opus 5.5 in six steps — Roger_M_Taylor · 2026-10-16
- Armin Ronacher's kid vibe-coded multiplayer with AI, teaching dad about PeerJS — mitsuhiko · 2026-10-16