Running Qwen3.8 Flash Next on dual RTX 3090: full llama.cpp config shared for tuning
ChopSticksPlease · reddit · 2026-09-12
A user shares their llama.cpp setup for Qwen3.8 Flash Next on dual RTX 3090s (48GB VRAM) + 128GB DDR4 + 40-core Xeon under Proxmox: 131072 ctx, q80 KV cache, flash attention, -ts 26,10, -ncmoe 26, and per-layer tensor overrides. Current results: PP 130-200 tps, TG 15 tps average; asking for tuning advice.
More from Infra
- TSMC reportedly raising CoWoS packaging prices up to 20% next year on AI chip demand — Beth_Kindig · 2026-10-31
- effect-harness on Cloudflare: run agent loops, tools and credentials on separate VMs — samgoodwin89 · 2026-10-31
- Google's Project Suncatcher boots TPU in orbit and beams back first AI response — sundarpichai · 2026-10-31
- Together AI launches Post-Training platform for agentic RL with tool-use environments — togethercompute · 2026-10-31
- General Compute raises $100M for inference capacity as CoreWeave stock hits 12-month low — YvesMulkers · 2026-10-31
- Analyst warns of unstable AI compute feedback loop as Kioxia's NAND price warning triggers flash crash — bookwormengr · 2026-10-31