Metal NVFP4/MXFP8 Kernels Cut Inkling 2 Mini Memory from 46.2GiB to 9.9GiB on M3 Max
GabGarrett · x · 2026-11-22
A notable local-AI improvement for Apple Silicon: Arthur Zucker added native NVFP4 and MXFP8 Metal kernels for Inkling 2 Mini to Hugging Face Transformers, letting its expert weights stay quantized in memory instead of being unpacked.
Result: on an M3 Max, memory usage drops from 46.2 GiB to just 9.9 GiB with identical logits and around 29.6 tokens/sec. This makes Inkling 2 Mini practical to run on 16GB and 24GB MacBooks — real progress for local AI.
More from Infra
- Uncensored 26.4B Inkling 2 Mini runs locally on a 12GB laptop, keeps 98% MMLU — udmrzn · 2026-11-22
- Training compute cost per task may be mere cents, argues 1a3orn — if models generalize from small runs — 1a3orn · 2026-11-22
- Miners 'hacked' Chutes for a year with millions in incentives to forge real end-to-end TEE privacy — const_reborn · 2026-11-22
- lxask: a fine-tuned 1.5B model turns plain English into offline Linux commands, self-checked against man pages — Arch-r007 · 2026-11-22
- Second OpenRouter Leak in a Month: MiniMax-Route Prompts May Have Been Read — const_reborn · 2026-11-22
- capy-flash v0.16 runs Inkling 2 Mini on two 16GB GPUs at 149 tok/s with 254k context in NVFP4 — FeydRowan · 2026-11-22