Why Local LLMs Are Still Hard to Use
zakadit · reddit · 2026-07-19
The post discusses "how to actually run 2T+ models." After listing their incredibly high-end local hardware, the author still finds models like Kimi K3 too slow, questioning what the "local AI victory" actually means for average users.
The core argument is that even with massive compute, the scale, inference speed, and context length of viable local models remain limited. Most people still use smaller-tier models, while super-sized models struggle to reach a "truly usable" speed locally.
More from Infra
- Meta approves nonvoting Class C shares as 2026 capex guidance hits $135B–$145B — thione · 2026-10-26
- Engineer with thousands of fabricated devices pours cold water on photonic crystal modulator yields — jwt0625 · 2026-10-26
- Anthropic S-1 reveals 48 cents of compute cost per dollar of Q2 revenue — nmasc_ · 2026-10-26
- Persistent AI joins UR network to route private AI inference with encrypted, IP-hidden traffic — 0xSammy · 2026-10-26
- Anthropic's disclosed compute commitments: $114.5B Google, $161.2B Broadcom top the list — BenBajarin · 2026-10-26
- SpaceX strikes $40B deal with Apollo Global to lease AI chips and data center hardware — Polymarket · 2026-10-26