QuixiAI open-sources SlimServe, the inference stack behind the 8x3090 Qwen deployment
QuixiAI · x · 2026-09-06
Eric Hartford open-sourced SlimServe on GitHub, the inference stack behind his 8x RTX 3090 deployment of Qwen3.8-Flash-Next. The repo (19,588 commits) includes vLLM integration, benchmark scripts, a perf worklog, the 3090 deployment blog writeup, plus engineering docs on GLM5.2 pipeline tracing, Metal scaling and quantization notes. Developers can reproduce the setup via the blog doc and benchmarks directory.
More from Infra
- PhD students in some countries get B200s on demand, unlike their US peers — HildeKuehne · 2026-10-23
- Nvidia's Berlin GTC spotlights humanoid robots and Vera Rubin AI supercomputing — emmanuelvivier · 2026-10-23
- Lenovo adds 5 AI Library use cases, claims 1-week deployment and up to 70% lower token cost — shashib · 2026-10-23
- Your kernel doesn't need to beat cuBLAS on every shape, argues GPU dev — blelbach · 2026-10-23
- llama.cpp team launches Llama: one-click local AI on your Mac menu bar — Hugging Face · 2026-10-23
- China's daily token use hits 500T, up 4x as average price doubles to ¥13.46 — teortaxesTex · 2026-10-23