Qwen3-8B on a rented A6000 measured at just 10 tokens/s
huangyun_122 · x · 2026-10-29
The author rented an A6000 on Runpod to run Qwen3-8B locally and measured only 10 tokens/s — a notably slow result for this hardware/model combination, a useful datapoint for single-GPU local deployment performance.
More from Infra
- Ollama's two-week model drop: LFM3-1.7B targets edge agents with 256K context at ~1GB — ollama · 2026-10-30
- Meta's GenRecs Tokenization lifts ad conversions +0.35% with semantic vocabulary — Meta_Engineers · 2026-10-30
- Parallel Language Models paper: multi-token generation at AR quality, up to 5.7x speedup — aaron_lou · 2026-10-30
- Oracle investor day: 1GW AI site earns ~$80B contract value at ~28% ROI, 3.6-year payback — rohanpaul_ai · 2026-10-30
- Sam Altman: Local inference is a terrible allocation of scarce global compute — morqon · 2026-10-30
- Dual RTX 4060 Ti local LLM benchmarks: 35B MoE decodes at 122.5 tok/s — Rooneybuk · 2026-10-30