RTX 3090 runs 27B LLM 4.6x faster via pure inference optimizations, no new hardware
rickasaurus · x · 2026-10-19
A r/LocalLLaMA user pushed a 2020 RTX 3090 running Qwen3.8 27B from 82 to 382 tok/s on a document-quoting task (4.6x) using only software tuning: MTP tweaks, a DFlash2 drafter plus a lookup drafter, and 16-token verify steps. Normal chat runs at 133 tok/s (still 1.6x faster) with context extended to 138K.
The poster reports similar gains on his own 4x 3090 rig: DeepSeek V4 got 29x faster, Flash prefill went from 433 to 1906 tok/s in one day, and a MiniMax H3 clip dropped from 11:21 to 3:45 — the GPUs had been idle-waiting 92% of the time. Core claim: local AI performance is a software problem, not a hardware problem.
More from Infra
- Qwen 3.8 Flash Next on 2x CMP 170HX hits 840 tok/s at 8 concurrency, full recipe released — dead-fish · 2026-10-19
- Google and Synaptics Preview Coral Microcontroller for On-Device AI at Under 6 Milliwatts — lmoroney · 2026-10-19
- Not Paris looters — just Series D founders who still can't afford GPUs — DesireeCachette · 2026-10-19
- Sovereign AI Blog ships an MCP server for its NVIDIA DGX Spark self-hosting log — modelcontextprotocol · 2026-10-19
- Why agent sandboxes became a thing: slow VMs pushed Lovable, E2B and Modal to build their own — vboykis · 2026-10-19
- Local Qwen vision model builds a 3D room game in 26 minutes on a single prompt — Beneficial-Ad-8127 · 2026-10-19