Five local models on DGX Spark: Qwen3.8-27B most reliable at 28/30, fastest models finish last
JeremyCMorgan · x · 2026-11-04
Boris Ivanov benchmarked five local models on an ASUS Ascent GX10 (GB10, 128GB unified memory) with his open-source llm-compile-bench harness: six coding tasks, five runs each, judged by cargo/g++/pytest/go test/tsc.
- Qwen3.8-27B (Q4KM, 31 tok/s): most reliable — 30/30 compiles, 28/30 tests pass, wall-to-pass 57s
- gpt-oss-120b (MXFP4): 26/30, fastest useful answers (27s), but its Go worker pool passed only 1/5 despite going 6/6 on the first run
- DeepSeek V4 Flash (91GB): 25/30 but slowest (169s), with runs dragging to 1600s+
- Qwen3.6-35B-A3B and Nemotron 3.5 Lightning (90 tok/s) finished last at 17/30 and 15/30
Key takeaway: single-run benchmarks mislead — reliability across runs and wall-to-pass time matter more than raw speed. Harness is open source.
More from Infra
- OpenAI builds Pando, a new Git backend that cut push latency ~80% serving ~1PB daily — timzaman · 2026-11-04
- AMD jumps 6.63% after hours on strong earnings, now a datacenter compute company — BenBajarin · 2026-11-04
- Electricity per AI image measured across RTX GPUs: ~3-10 cents per 1,000 images with FLUX.2 klein — strata2signal · 2026-11-04
- SemiAnalysis: SpaceX to Become World's Biggest AI Cloud With 13GW in 1.5 Years — dylan522p · 2026-11-04
- ORNL Paper: Useful Quantum Computing Needs 4M Qubits and 800 Square Meters — anirbanbandyo · 2026-11-04
- Snowflake open-sources Arctic Platform, a modular post-training framework for LLMs — StasBekman · 2026-11-04