Local Qwen 3.8 Flash Next benchmark jumps from 91 to 98/100 with AWQ quants and patched vLLM
WonderRico · reddit · 2026-09-05
Blogger WonderRico updated his local LLM benchmark recipe for Qwen 3.8 Flash Next, raising the score from 91 to 98/100.
- New stack: AWQ W4A16 weights plus INT4 n-gram PLE quant (both on Hugging Face), served via vLLM patched with files from the primitive-ai repo
- Goal: load both weights and PLE in 4-bit on sm89/sm120 GPUs; slower than the previous patched SGLang setup but a large quality jump
- Hit 98/100 at both medium and xhigh reasoning — best score and most efficient combo on his bench
- He'll investigate whether the delta comes from the engine patches or the quants; graphs and data are public
More from Infra
- PostgreSQL 19 slips to November as LLM-assisted code review floods committers with edge cases — sanjaykalra · 2026-10-21
- Decision Models' routing classifier hits AntSeed, cutting inference cost to 24% of always-Opus — airesearch12 · 2026-10-21
- Eon cuts engineer onboarding from 2 days to 2 minutes with an MCP Gateway on Bedrock AgentCore — AWS ML Blog · 2026-10-20
- a16z's Ben Horowitz says AI buildout isn't the 90s bubble — an investor pushes back — henloitsjoyce · 2026-10-20
- Micron and Nvidia drive ~36% of S&P 500's Q3 earnings growth — Beth_Kindig · 2026-10-20
- NVIDIA lands in Europe: Jensen Huang and German chancellor discuss industrial AI at GTC Berlin — nvidia · 2026-10-20