Dual 3090s running Qwen3.8-27B INT4 is significantly faster than FP8
Travnewmatic · reddit · 2026-10-29
A Redditor shares a local LLM setup: on 2x RTX 3090s, switching from dual vLLM Qwen3.8-27B FP8 to RedHatAI's INT4 quantization delivered a major speedup — far better than early INT4 builds. Full QEngine compose configs and links are provided for reproduction.
More from Infra
- Oracle investor day: 1GW AI site earns ~$80B contract value at ~28% ROI, 3.6-year payback — rohanpaul_ai · 2026-10-30
- Sam Altman: Local inference is a terrible allocation of scarce global compute — morqon · 2026-10-30
- Dual RTX 4060 Ti local LLM benchmarks: 35B MoE decodes at 122.5 tok/s — Rooneybuk · 2026-10-30
- Brown University report: US electricity law makes it nearly impossible to unplug a rogue AI — relianceschool · 2026-10-30
- AI buildout could push data-center power demand to 950 TWh by 2030: IEA — PeterDiamandis · 2026-10-30
- Morph and InferenceNet drop optional args in DeepSeek tool calls on OpenRouter — emilahlback · 2026-10-29