What Models Are People Running on 192GB Machines?
CentrifugalMalaise · reddit · 2026-07-19
A user with 192GB of RAM shares their experience running local LLMs.
They primarily run the Unsloth Dynamic UD-Q3KXL gguf for Qwen3.5-397B and mention using Claude to fix some llama.cpp issues related to hybrid recursive models, including:
- Fixing KV cache invalidation when vision is enabled
- Fixing instant recovery after saving session slots to disk
- An ongoing issue where tool calling breaks the KV cache, which they plan to fix next
They also list other LLMs they are interested in trying: GLM 4.7 357B, Deepseek V4 Flash 284B, Tencent Hy3 295B, Minimax M3 428B, Laguna M.1 225B, Minimax M2.7 229B, and Mimo 2.5 310B, while asking the community what others are running and how these compare to Qwen3.5-397B.
More from Infra
- Resin lets an agent build standalone LLM inference engines; vLLM and SGLang optimize DeepSeek kernels differently — AccBalanced · 2026-10-27
- Local benchmark: Flint matches Qwen3.6 quality at 50/52 while running 3.6x faster on M4 Pro — RichardSocher · 2026-10-27
- Anthropic S-1: CSP marketplace revenue jumped from 11% to 48% in three years — AccBalanced · 2026-10-27
- Alchemy adds effect/Workflow support across Durable Objects, Workflows and Lambda Durable Functions — samgoodwin89 · 2026-10-27
- Alchemy's Lambda Web Functions run plain Node web servers on Lambda, no OpenNext needed — samgoodwin89 · 2026-10-27
- Alchemy Cuts Cloudflare Bindings CPU Time by Up to 166× Via Profiling — samgoodwin89 · 2026-10-27