Running Qwen3.8 Locally on 6x3090s with exllamav3 Hits 80-120 tok/s
takoulseum · reddit · 2026-09-26
A redditor shares a local-agent setup: Qwen3.8 in 6bpw exl3 quant (by turboderp) served via the exllamav3 engine on 6x RTX 3090s delivers roughly 80-120 tok/s with good prefill and quality — "crazy for CUDA." They control the Hermes agent daily from their phone over Matrix, and use it beyond delegating to coding harnesses like opencode, assuming lower quants on fewer GPUs would work similarly.
More from coding & agent
- Stealth Model Space Bunny Builds Interactive 3D Flight Tracker in 3.5 Minutes — socialwithaayan · 2026-09-26
- OpenAI Incident Report: Agents Escalated Privileges via Artifactory Using a Shared Key — tarantulae · 2026-09-26
- OpenAI incident report: agents escalated privilege via Artifactory using a shared key — tarantulae · 2026-09-26
- Dev burns 188 billion tokens processing 47k issues with an autonomous agent over a weekend — jamesbrooksco · 2026-09-26
- Fine-tune your own GLiNER model for under $10 with a Claude Code/Codex agent skill — max_paperclips · 2026-09-26
- Dev Builds Van Gogh Scene Generator with Codex: Research Existing Three.js Projects First, Then Iterate — simonxxoo · 2026-09-26