Qwen3.8-Flash-Next on SGLang NVFP4: 254K-Context TTFT Drops 35s→22s, Full Gauntlet Tested
FantasticNature7590 · reddit · 2026-09-17
- On a single RTX PRO 6000 Blackwell (96GB), the author benchmarked Qwen3.8-Flash-Next with SGLang's official NVFP4 image: TTFT on a 254K-token prompt fell from 34.9s to 22.4s, prefill rose from 7,284 to 11,352 tok/s (1.56×), and cached repeat prompts dropped TTFT from 21.7s to 0.44s (49×).
- Capability results: long-context recall 81/81 (fully correct even at 99% window fill); BFCL single-turn tool accuracy 85.2% vs 49.5% multi-turn; τ²-bench telecom 68.1% completion but only 41.2% passed all three attempts; battle gauntlet 13/17 wins with thinking on, 8/17 off.
- SVG, video-editing and animated-design generations mostly passed hard checks (video editing scored 10/10 on the author's rubric). Full hardware config, methodology and latency/throughput tables across context lengths included — a rare system-level long-context benchmark on an open stack.
More from Infra
- OpenRouter crosses 10 quadrillion annualized token run rate, doubling every 10.5 weeks — deedydas · 2026-11-02
- Palmer Luckey asks: can you use datacenter RAM for things prohibited by the operator's terms of service? — avaitopiper · 2026-11-02
- Disable MTP for 20-30% faster local LLM inference on limited RAM — InternalMode8159 · 2026-11-02
- Kioxia up 216% in a year but down 68% from its June 2026 peak — zephyr_z9 · 2026-11-02
- Intel plunges 29%, worst day since the 1970s, wiping $12B off US government stake — emmanuelvivier · 2026-11-02
- Researcher hoards nearly 1TB of DRAM, refuses to sell a stick to data centers — kevinnbass · 2026-11-02