Vulkan vs CUDA Benchmarks for Qwen3.6-35B: Even on a 3060, 5-30% Slower on Datacenter GPUs
Serious_Scarcity7818 · reddit · 2026-11-13
RustingStrata, a pure-Rust MoE inference engine whose CUDA backend compiles kernels at runtime via NVRTC (no nvcc, no C++ build step), benchmarked the same Qwen3.6-35B-A3B GGUF model (fully in VRAM, MTP off) on five NVIDIA cards across its CUDA and Vulkan backends, mean of 3 runs:
- RTX 3060 12GB: Vulkan parity (97% prefill / 100% decode)
- L40S: 95% / 87%; A30: 94% / 79%; A100 80GB: 90% / 76%; H100 NVL: 70% / 78%
- On datacenter cards Vulkan loses 5-30% prefill and 13-24% decode, with noisier runs (A100 4K prefill ranged 4788-5990 vs 6186-6214 on CUDA)
- Output consistency: greedy top-1 agreement with llama.cpp over 20 prompts × 32 tokens is 98-99% on both backends
Reproduction command and repo link included.
More from Infra
- Unsloth ships GGUF quant of Inkling-2-Small for local inference — gnukeith · 2026-11-13
- No fp8 on Ampere: maker ships int8 Krea 2 Turbo build for 20/30-series GPUs — in_habitants · 2026-11-13
- GlobalFoundries photonics sold out, $1B revenue target pulled forward amid NPO/CPO shift — BenBajarin · 2026-11-13
- foldz: lossless GGUF compressor saves 8.66% across 9 Hub models — Simpli-curious · 2026-11-13
- Nemotron Bolt hits 70 tok/s on a 3090, inspiring MTP tuning on Qwen — samajhdar-bano2 · 2026-11-13
- Cerebras Q3 revenue hits $231M with 700MW of data center capacity contracted — BenBajarin · 2026-11-13