Unsloth Releases Inkling-2-Small GGUF Quants, Running 594B Model Locally in 171GB
Unsloth published GGUF quantized weights and a local guide for Thinking Machines Lab's Inkling 2 Small, a 594B-parameter MoE, making it runnable locally with just 171GB via 1-bit dynamic quantization.
2026-11-13 ~ 2026-11-13 · 2 related posts
- Episode 1: Thinking Machines Open-Sources Inkling 2 Small: 594B MoE Weights Under Apache 2.0(2026-11-13, 27 posts)
- Episode 2: vLLM adds Thinking Machines' Inkling-2-Small with 1M context and 1100 tok/s(2026-11-13, 2 posts)
- Episode 3: Thinking Machines' Inkling 2 Small lands on Ollama with 1M context(2026-11-13, 2 posts)
- Episode 4: Thinking Machines' Open-Weights Inkling Small 2 Wins Praise(2026-11-13, 2 posts)
- Episode 5: Unsloth Releases Inkling-2-Small GGUF Quants, Running 594B Model Locally in 171GB(2026-11-13, 2 posts)
- Unsloth ships GGUF quant of Inkling-2-Small for local inference — gnukeith · 2026-11-13
- Unsloth quantizes Inkling 2 Small: 594B MoE runs locally at 171.7GB in 1-bit — gnukeith · 2026-11-13