Why Is ChatGPT Instant but Local Qwen Slower? Reddit Debates Inference Stacks
dhtp2018 · reddit · 2026-10-19
Reddit user dhtp2018 compared a locally run Qwen 3.8B with Copilot/ChatGPT: the cloud service output nearly a full page almost instantly and solved the same problem with noticeably fewer tokens, while the local model was slower and less token-efficient.
The poster wonders whether local/Chinese models lack some architectural capability or whether enterprise GPU hardware explains the gap. The discussion points to cloud-side inference stack optimizations — speculative decoding, KV cache management, batching — rather than architecture differences.
More from Models
- Claude now happily decompiles anything after lecturing users 6 months ago — competition with DeepSeek cited — AccBalanced · 2026-10-19
- GPT-6-Astra report: robot benchmarks near saturation with 10x execution budget — ccloy · 2026-10-19
- Deep research veterans see value in instant LLM summaries that novices miss — generativist · 2026-10-19
- Waze support confirms no Gemini integration as Pixel voice activation fails — Particular_Ask_1453 · 2026-10-19
- ChatGPT users report randomly deleted turns and spurious conversation-length limit errors — Aggressive_Case8035 · 2026-10-19
- Andrew Carr: it took 3 tries to convince Argon it could output 1M tokens — andrew_n_carr · 2026-10-19