Qwen 72B runs on RTX 5090 at 115 tokens/sec, BF16 weights 55.6GB
AccBalanced · x · 2026-08-18
Demonstrates running Qwen 2.5 72B (possibly mislabeled as 3.8-27B in original tweet, likely 72B or 32B given the context) locally on an RTX 5090 (32GB VRAM) system, achieving 115 tokens/sec. Notes that the official BF16 checkpoint for Qwen 2.5 72B is 55.6 GB.
More from Models
- One year of Recursive Language Models: GPT-5-mini beats GPT-5 by 110% on 132k-token tasks, cheaper too — a1zhang · 2026-10-16
- Prediction: Opus 5.5-level models on laptops by 2027, backed by Qwen 27B data — cephaloform · 2026-10-16
- LLM-as-a-Verifier lifts Nemotron 3.5 Super by 9-10 points on agentic benchmarks — simonguozirui · 2026-10-16
- Bonsai 2 ternary models tested: 4B is the sweet spot, but 27B unexpectedly fumbles file writes — cephaloform · 2026-10-16
- Meta's Omnilingual ASR, covering 1,600 languages, named a TIME Best Invention of 2026 — AIatMeta · 2026-10-16
- Apollo Research: GLM-5.3 shows unprecedented 'update-aware reasoning' — repligate · 2026-10-16