Self-made W8A16 quant of Qwen 3.8 27B hits ~100 tok/s on 2x RTX 3090
TacGibs · reddit · 2026-10-11
A user released their own 8-bit AWQ + GPTQ quant (W8A16) of Qwen 3.8 27B Swift 1.5, claiming it's the best speed/quality tradeoff for 48GB VRAM:
- Runs with vLLM + MTP (MTP=5) on 2x RTX 3090 at nearly 100 tok/s decode, 21.6GB per card with 128k context and FP16 KV cache;
- The author reports it 'overthinks' less than the original 3.8 27B;
- Open-sourced on Hugging Face (TacGibs/Swift-1.5-Qwen3.8-27B-W8A16), quantized on 4x 3090s with details in the model card.
More from Infra
- OpenJet claims Qwen3.8-27B coding runs 1.9x faster than llama.cpp on one RTX 3090 — JeremyCMorgan · 2026-10-23
- Jensen Huang lays out NVIDIA's full Vera Rubin agentic system: NVL72, MGX, CMX and RTX — rohanpaul_ai · 2026-10-23
- ONNX to Transformers.js: running models directly in your browser — JosephJacks_ · 2026-10-23
- MiniMax M3.1 Flash free for 3 days on 0G, with 1M context and 512K output — MiniMax_AI · 2026-10-23
- "We bet on Cloudflare and it paid off": R2 and D1 at billion scale — dinasaur_404 · 2026-10-23
- Public data is drying up; private training data is turning into a browsable catalog — SucceededMind · 2026-10-23