Running Qwen3.8 Locally on 6x3090s with exllamav3 Hits 80-120 tok/s
takoulseum · reddit · 2026-09-26
A redditor shares a local-agent setup: Qwen3.8 in 6bpw exl3 quant (by turboderp) served via the exllamav3 engine on 6x RTX 3090s delivers roughly 80-120 tok/s with good prefill and quality — "crazy for CUDA." They control the Hermes agent daily from their phone over Matrix, and use it beyond delegating to coding harnesses like opencode, assuming lower quants on fewer GPUs would work similarly.
More from coding & agent
- Sydney Codex Community Buildathon set for Nov 14 with Cerebras-backed ultrafast compute track — paw_lean · 2026-11-11
- shadcn's improve hits 10k stars: let top models plan, cheap models execute code fixes — shadcn · 2026-11-11
- OpenAI open-sources Codex Security CLI and SDK for AI-driven vulnerability scanning and fixing — openai · 2026-11-11
- Claude Opus 5.5 runs an architecture search and finds AttnRes-Canon, a new top Transformer design — SonglinYang4 · 2026-11-11
- 5 research habits worth knowing: read papers backwards, reproduce first, wire coding agents via MCP — cneuralnetwork · 2026-11-11
- 'Adam can now do CAE': AI agent showcase in engineering simulation draws buzz — burhop · 2026-11-11