Qwen 27B on RTX 5070 Ti laptop: 4.5 tok/s with 80% MTP acceptance
CoffeeToCode99 · reddit · 2026-08-15
Running Qwen3.8-27B (UD-Q4KXL) on a laptop with a 12GB RTX 5070 Ti GPU, utilizing CPU offloading and MTP speculative decoding, achieved approximately 4.5 tok/s with an MTP acceptance rate around 80%. Tests show stable performance for factual queries and Python coding, making it a viable setup prioritizing quality over speed.
More from Infra
- Three CoreWeave co-founders have sold $3.3B in shares since IPO amid credit worries — SumitGup · 2026-10-16
- Cherry pi on a Raspberry Pi: the tiny-model rumor developers are watching — sull · 2026-10-16
- CoreWeave's debt costs falling: DDTL spreads tightened from S+9.62% to S+2.25% — BenBajarin · 2026-10-16
- SoftBank lines up US IPOs for SB Energy and AI robotics spinoff Roze, valuations top $50B — nordicinst · 2026-10-16
- TRL reproduces Modal's mixed-precision async RL: plain GRPO collapses, SC+MIS holds 0.72 — QGallouedec · 2026-10-16
- Hugging Face scales Indic pretraining with 2.8T tokens and an open data pipeline — Hugging Face Blog · 2026-10-16