Self-Hosting Qwen3.8-27B NVFP4 on SGLang Hits 200+ Tokens/Sec
gnukeith · x · 2026-08-15
Developer @gnukeith keeps expanding his self-hosted stack: Vaultwarden, SGLang, Qwen3.8-27B NVFP4, Open WebUI, Forgejo, FUTO Notes Sync, and Immich.
The highlight: SGLang serving Qwen3.8-27B NVFP4 runs at roughly 200+ tokens per second, making local inference feel smooth.
More from Infra
- CoreWeave's debt costs falling: DDTL spreads tightened from S+9.62% to S+2.25% — BenBajarin · 2026-10-16
- SoftBank lines up US IPOs for SB Energy and AI robotics spinoff Roze, valuations top $50B — nordicinst · 2026-10-16
- TRL reproduces Modal's mixed-precision async RL: plain GRPO collapses, SC+MIS holds 0.72 — QGallouedec · 2026-10-16
- Hugging Face scales Indic pretraining with 2.8T tokens and an open data pipeline — Hugging Face Blog · 2026-10-16
- NVIDIA Vera Rubin NVL72 hits up to 5.7x GB300 throughput per MW on agent benchmark — ArtificialAnlys · 2026-10-16
- 14 open-source LLM observability tools to know in 2026, from Langfuse to Laminar — TheTuringPost · 2026-10-16