Unsloth ships GGUF quant of Inkling-2-Small for local inference
gnukeith · x · 2026-11-13
Unsloth published unsloth/Inkling-2-Small-GGUF on Hugging Face, providing GGUF-quantized weights of Inkling-2-Small so it can run locally via llama.cpp-style stacks. Author gnukeith thanked the Unsloth team (@UnslothAI, @danielhanchen).
The page text also includes the model's Jinja chat-template macros: float formatting (scientific notation vs. decimal placement), falling back to float parsing for integers outside the int64/uint64 range, and canonical JSON dumping with code-point-sorted keys, compact separators and raw UTF-8.
More from Infra
- Cloudflare hosts Thinking Machines' inkling 2 small on Workers AI at day-0 speed — michellechen · 2026-11-14
- Liquid-Codec decoder ported to Apple Neural Engine, 6x faster at 316x real time — JosephJacks_ · 2026-11-14
- AMD MI355X with FlyDSL kernels beats Blackwell running MiniMax M3.1 in vLLM — AnushElangovan · 2026-11-14
- Tencent paper: pruning neurons by routed contribution keeps 91-98% MoE performance — rohanpaul_ai · 2026-11-14
- AMD's hrx-loom stack: agent-written kernels beat llama.cpp across 51 LLMs on Strix Halo — jfowers_amd · 2026-11-14
- Microsoft Cloud tops $63B as AI buildout keeps scaling, says Nadella — thedealdirector · 2026-11-14