SenseNova-Loop-9B official GGUFs land: 5.8 GB at Q4, 256K context in ~15 GiB
Binary_orchid · reddit · 2026-11-18
Official quants for SenseNova-Loop-9B, the 9B Looped Transformer released this week, are now on Hugging Face:
- GGUF: Q4KM (5.8 GB), Q5KM, Q80 and BF16, plus a 0.9 GB mmproj for image input.
- Compatibility: the sensenovaloop architecture isn't in mainline llama.cpp yet (issue #31571), so it runs on the official fork OpenSenseNova/llama.cpp: llama serve -hf sensenova/SenseNova-Loop-9B-GGUF:Q4KM -ngl 99 -c 4096 --port 8000.
- AWQ: a 4-bit AWQ is also available with a matching MTP draft model in the same repo.
- Memory: 4-bit weights plus 8-bit KV cache total 10 GiB for 64K context and 15 GiB for 256K.
More from Infra
- Turn an 8GB old laptop into a private offline AI in 20 minutes with Jan and Gemma — alex_verem · 2026-11-18
- Deep dive: the full rise and fall of DEC's Alpha processor architecture — ArtificialOther · 2026-11-18
- Nightmare Mobile Runs FLUX, Qwen Image and Wan 2.2 Fully Offline on Phone NPUs — Few_Caregiver8134 · 2026-11-18
- Free online book: LLM Inference Handbook for production deployment — burkov · 2026-11-18
- SEC opens civil probe into Nvidia over undisclosed China-linked chip sales — The Decoder · 2026-11-18
- Open-source LLM Optimization Handbook: 48 topics with hands-on quantization, KV cache and kernel problems — udmrzn · 2026-11-18