Inkling-2-Mini runs on laptops with as little as 9GB RAM, per Unsloth local guide
NielsRogge · x · 2026-11-21
NielsRogge points to a new UnslothAI guide for running Thinking Machines Lab's Inkling-2-Mini locally on a laptop, noting it needs as little as 9GB of RAM depending on quantization size.
Memory requirements by quantization level:
- 1-bit: 9 GB
- 2-bit: 10-11 GB
- 3-bit: 13-15 GB
- 4-bit: 17-18 GB
- 8-bit: 24 GB
Higher-bit versions perform better but use more memory and can slow inference. The author personally runs the 3-bit version on his MacBook.
Related event: Unsloth Guide: Run Inkling-2-Mini Locally with Just 9GB RAM(2 posts)→
More from Infra
- Only when infra resources burn out will the application layer see its spring — yangyi · 2026-11-21
- 26 LLM routers caught injecting malicious tool calls, draining a client's $500k wallet — 0xkarasy · 2026-11-21
- OpenRouter accused of sending users' BYOK private keys to China despite earlier denials — 0xkarasy · 2026-11-21
- OVH's Olala Mini 7A1B: 360 tok/s decode on one RTX 5090, 96K context, v2.0 to exceed 100B — JiliJeanlouis · 2026-11-21
- If you bet the century on AI, you need the financial and physical supply chain to back it — we don't — ZeroStateReflex · 2026-11-21
- NInfer-all adds Inkling 2 Mini, nearly 2x llama.cpp decode speed on RTX 5090 — WaveCut · 2026-11-21