ZGX Fury stress-tested: 351 tok/s on Qwen3.8-Flash-Next, offline agents and local RAG demos
Sam Witteveen · youtube · 2026-11-16
YouTuber Sam Witteveen publishes a hands-on deep dive (HP/NVIDIA sponsored) pushing the HP ZGX Fury workstation to its limits.
Highlights (with timestamps):
- Memory math: HBM vs unified memory, whether a model fits
- Inference fundamentals: prefill vs decode, speculative decoding, concurrent users, and why agents need faster tokens
- Measured speeds shown in the thumbnail: Qwen3.8-Flash-Next 351 tok/s, Nemotron 3.5 Super VL 496 tok/s, GLM 5.3 Flash 283 tok/s, DeepSeek V4 Flash 226 tok/s
- Workloads: offline coding agents, private local RAG, audio/video transcription, video generation
A useful performance reference for anyone into local deployment and on-device inference.
More from Infra
- Training-free page-sparse decode in llama.cpp: 1.72× faster 128k decoding on an RTX 5090 — topppggg · 2026-11-16
- llama.cpp offloads CI caches and runners to Hugging Face to survive 35 PRs/day — ggerganov · 2026-11-16
- India's Sarvam launches Token Factory, hosting top open models on local infra — HarveenChadha · 2026-11-16
- Gemini 3.5 Flash now runs on air-gapped Google Distributed Cloud, offline up to 12 months — shashib · 2026-11-16
- Gemma.c: full Gemma 1/2/3 multimodal inference in a single C file, ~72 tok/s on CPU at INT8 — This-Purchase-3325 · 2026-11-16
- BIS Issues Interim Final Rule Addressing Remote Access to Advanced AI Chips — apples_jimmy · 2026-11-16