llama.cpp PR Runs Missed MoE Experts on CPU, Big Speedup for Low-VRAM Offloading
jacek2023 · reddit · 2026-11-08
llama.cpp PR #30966 by am17an delivers a significant speedup for MoE offloading: experts missed from VRAM are executed on CPU, with a tuning script included — a major win for low-VRAM local inference, similar in spirit to Strata's tuning mechanism.
More from Infra
- Hugging Face launches live training streams, starting with RL-tuned LoRA adapters — art_zucker · 2026-11-08
- Lobsters thread: how AI traffic is degrading the web for human developers — vboykis · 2026-11-08
- Oracle's quarterly capex up 2,540% to $28.5B in five years, fastest growth among hyperscalers — Beth_Kindig · 2026-11-08
- Decentralized Compute Network Lium Hits $2.32M GPU Spend in October, Up 74% MoM — markjeffrey · 2026-11-08
- Lex Sokolin: Google wants a piece of AI datacenters in space — minchoi · 2026-11-08
- Megaspeed exec recruited to spy on Nvidia CEO, US intelligence memo reportedly claims — TechCrunch AI · 2026-11-08