ik_llama.cpp Fork Update: Matches Strata Speeds on a 6GB RTX 2060
IceFog72 · reddit · 2026-10-25
Developer IceFog72 shipped an update to the ikllama.cpp fork, continuing the MoE expert-residency optimization idea with a different implementation, finally matching Strata's inference speeds on a 6GB VRAM RTX 2060.
The code still needs cleanup and has only been thoroughly tested on a single 2060, so the author isn't attempting an upstream merge yet due to lack of feedback; details are on the GitHub repo, with testing and feedback welcome.
More from Infra
- Samsung reportedly using TSMC 3nm for Nvidia's next-gen NVHBM custom chip base dies — Beth_Kindig · 2026-10-25
- No AI bubble yet: revenue per GPU minus rental is the real equation — PTrubey · 2026-10-25
- AI labs are flipping GPU contracts: H100/B200 rental prices up 2-3x since early lock-ins — Yuchenj_UW · 2026-10-25
- Swarms open-sources RouteHub, an LLM gateway that imports 190x faster than litellm — KyeGomezB · 2026-10-25
- Radeon 9700 Prices Double in a Month, Hurting Personal Inference Server Builds — UltramanQuar · 2026-10-25
- New blog demystifies GPU communications, from page tables to NVLink, via SGLang internals — BanghuaZ · 2026-10-25