Running Qwen 3.8 flash with 256K context on a 12GB GPU: open-source local setup hits 30-38 tok/s
Present_Resource4022 · reddit · 2026-10-14
A Reddit user open-sourced the Gyro-M experimental setup (repo: GyroStrata), running Qwen 3.8 flash with full 256K context on a 4070 Super 12GB + 32GB RAM, hitting 950-1800 tok/s prefill and 30-38 tok/s output.
Key points:
- The repo ships Strata patches, tuned settings, a model downloader, installer and launcher
- MTP and vision are deliberately disabled (they hurt speed in testing); 16GB VRAM users may try enabling MTP
- Speeds depend heavily on CPU, SSD, RAM speed and GPU bandwidth; the author includes a prompt for Codex/Claude Code to auto-tune configs for your own hardware
- Unlike DGX Spark or dual-3090 recipes, heterogeneous low-VRAM setups need per-machine tuning
More from coding & agent
- Agent Teams Cost Up to 5.1x More but Barely Beat Single Agents, Vals AI Finds — josh_wills · 2026-10-15
- BaitBench: frontier agents take data-leaky shortcuts in 57% of runs and mostly know they're cheating — gleech · 2026-10-15
- Claude Opus spots bcrypt-per-request API auth behind VPS CPU spikes, load drops 0.9 to 0.4 — DanielLockyer · 2026-10-15
- Burkov: labs are now training coding LLMs on git history, exactly as he predicted 9 months ago — burkov · 2026-10-15
- Cohere research: swarms of small models reach frontier-level software engineering — Cohere_Labs · 2026-10-15
- 12 GitHub repos to supercharge Claude Code: context, parallel sessions, automation — Roger_M_Taylor · 2026-10-15