Go-LLM repo details running Flash-Next on dual RTX 3090 via vLLM, with honest benchmarks
Motor_Ad16 · reddit · 2026-09-15
A new open-source repo, Go-LLM, packages working notes for running Qwen3.8-Flash-Next with vLLM: the official path (vLLM nightly + Blackwell hardware + NVFP4 checkpoints) and a local workaround on dual RTX 3090 using a GGUF-capable 0.29.0 fork. Stable IQ4XS quantization delivers 36.5 tok/s at 1k decode and 19 tok/s at 126k, while Q4KXL stays experimental due to long-context instability. The author also retracts an earlier non-reproducible 6665 tok/s prefill figure.
More from Infra
- To dethrone Postgres, new databases must prioritize ecosystem integrations from day one — mgill25 · 2026-11-02
- ESP32-powered 4-inch touchscreen builds a TUI dashboard for monitoring a DGX inference cluster — CATLLM · 2026-11-02
- OpenRouter crosses 10 quadrillion annualized token run rate, doubling every 10.5 weeks — deedydas · 2026-11-02
- Palmer Luckey asks: can you use datacenter RAM for things prohibited by the operator's terms of service? — avaitopiper · 2026-11-02
- Disable MTP for 20-30% faster local LLM inference on limited RAM — InternalMode8159 · 2026-11-02
- Kioxia up 216% in a year but down 68% from its June 2026 peak — zephyr_z9 · 2026-11-02