Unsloth guide: run Inkling 2 Mini (26.4B MoE) locally, 1-bit needs just 9GB RAM
NielsRogge · x · 2026-11-21
Unsloth published a full local-run guide for Thinking Machines Lab's Inkling 2 Mini, a 26.4B MoE (5.4B active) model under Apache 2.0, built on a Looped Transformers variant with 1M context for coding, agentic, and vision workloads.
Key facts:
- Benchmarks: 48.2% on DeepSWE v1.1 and 77.9% on SWEBench Pro v2 (Hard) — strongest at its size, ahead of Qwen3.8-27B; ties it on Terminal Bench 4.0 at 5.6%
- Memory: 1-bit 9GB → 8-bit lossless 24GB; recommended quant UD-Q4KXL (17.6GB), full precision needs 16-24GB RAM
- Runs on Mac, NVIDIA, and AMD via Unsloth Desktop (less memory, faster inference) or llama.cpp with looped-layer support and per-expert NVFP4 scales
- Recommended settings: Temperature 1.0, Top-p 1.0, Top-k 0, Min-p 0.0
- Vision projector mmproj-F16.gguf adds 639MB; examples use 32K context, reducible if memory-constrained
Related event: Unsloth Guide: Run Inkling-2-Mini Locally with Just 9GB RAM(2 posts)→
More from coding & agent
- Dev used Claude + his own LinkedIn MCP server for outreach: 30 DMs, 3 paying users, $63 MRR — IHAVENOSALT · 2026-11-21
- RunWeave: open-source chaos harness injects rate limits, 503s and broken tool calls into agents — MrinmayDas · 2026-11-21
- MCP server for a medical icon library lets Claude fetch hand-drawn SVGs by clinical terms — blogoodf · 2026-11-21
- Agent completes full business tax return in 30 minutes at ~1% of human cost — intellectronica · 2026-11-21
- Multica launches a PM workspace to manage human + agent coding teams — jiayuan_jy · 2026-11-21
- Antigravity logs reveal "region unavailable" was really security verification; Google opens subscriptions Dec 2 — lxfater · 2026-11-21