Quantized Inkling Runs on Old Laptops, Metal Inference 5.8x Faster After Requantization
Unsloth's Inkling UD-Q8KXL quantization combines BF16 dense weights with NVFP4 experts, and requantization yields a 5.8x Metal inference speedup. Users report running it locally even on five-year-old laptops.
2026-11-21 ~ 2026-11-21 · 2 related posts
- NVFP4 MoE experts force CPU fallback in llama.cpp; requantizing boosts Metal speed 5.8x — teortaxesTex · 2026-11-21
- Running quantized Inkling on a 5-year-old laptop: pre-AGI perks go off-grid — teortaxesTex · 2026-11-21