Single DGX Spark Decodes 15K Tokens from 124B Model with Stable Throughput
AcanthisittaOk1699 · reddit · 2026-08-14
A developer tested the community Q5 GGUF quantization of Ling-3.0-flash (124B parameters) on a single DGX Spark. Given a 33-token prompt, the model generated a massive 15,128 tokens in a single response, taking about seven minutes.
The most notable observation is the decoding throughput stability. Even as KV cache accumulated over 15,000 tokens, the decode speed remained remarkably flat, barely shifting from 35.62 tok/s to 35.68 tok/s throughout the entire generation process.
Related event: Single DGX Spark Tests 124B Model with High-Speed Decoding(2 posts)→
More from Infra
- New hardware day: local LLM deployment the moment it arrives — lxfater · 2026-10-16
- Together AI's score centering fixes RL training collapse under quantized inference — nrehiew_ · 2026-10-16
- Inference is splitting prefill and decode across specialized chips, not just H100s — AccBalanced · 2026-10-16
- Rebasely launches in beta: Git hosting built for humans and coding agents — handotdev · 2026-10-16
- Drama3 TTS lands in audio.cpp: up to 5x real-time on RTX 5090, <8GB VRAM mode too — Acceptable-Cycle4645 · 2026-10-16
- Three CoreWeave co-founders have sold $3.3B in shares since IPO amid credit worries — SumitGup · 2026-10-16