Inkling 2 Mini quantized to 13.9GB runs at ~60 t/s decode on a single RTX 4090

indrasmirror · reddit · 2026-11-21

A developer got Thinking Machines' open-weights Inkling 2 Mini — a 26B looped MoE (3B active, 12 core layers run 3 passes per token) — running on one RTX 4090 after abliteration and quantization to a 13.9GB GGUF.

Weights on Hugging Face, engine on GitHub.

Original post →

More from Infra

Infra channel →