Running a 100B+ Qwen3 model locally on 64GB RAM: good vibes, short 192K context
lxfater · x · 2026-10-04
The author deployed a quantized 100B+ parameter Qwen3 model locally on a 64GB RAM machine. First impressions are positive — the new UI looks noticeably better — with the only drawback being the 192K context window. They're now looking for real use cases for their new local powerhouse.
More from Infra
- Ex-Waymo exec Abe Murray: CUDA and ISAs won't stay moats, huge chip innovation incoming — IgorCarron · 2026-11-20
- MIT and NVIDIA's QDA uses agents to auto-design LLM quantization recipes, beating top public ones by up to 2.2 points — omarsar0 · 2026-11-20
- Caixin: US remote-access cutoff and Nvidia CEO exit leave China's tech giants halting overseas compute deals — Gregory_C_Allen · 2026-11-20
- QuixiInfer: Rust Engine Runs 125B MoE Model on a Single RTX 5090 — QuixiAI · 2026-11-20
- QuixiInfer: from-scratch Rust engine runs DeepSeek with 256k context on a single RTX 5090 at ~30 tok/s — QuixiAI · 2026-11-20
- AMD's Lisa Su: data center to hit ~80% of revenue by 2027, sees $220B server CPU market — AccBalanced · 2026-11-20