Raon-1 Nano 3.1B gets native ik_llama.cpp runtime with 17% faster MTP speculative decoding
Any_Bus9205 · reddit · 2026-11-12
KRAFTON released Raon-1-nano, a 3.1B dense hybrid model (18x Raon Delta Attention + 6x NoPE gated MLA + 1 MTP block, 4-stream mHC, per-layer embeddings), with official GGUF weights; upstream llama.cpp support is still pending.
A new PR (#2903) on ikllama.cpp adds full native runtime plus an HF-to-GGUF converter:
- Complete RDA, NoPE MLA, mHC and PLE implementations; SwiGLU FFN forces F32 accumulators to avoid FP16 overflow/NaN on CUDA; output gate fixed to sigmoid, fixing garbage output from the default SiLU
- Native MTP speculative decoding via --spec-type mtp:nmax=1,pmin=0.0
Verified on CPU-only hardware (i5-13420H, AVX2+VNNI) with the official Q4KM checkpoint (2.32 GiB): perplexity 1.0976, essentially matching the KRAFTON fork; at 4 threads, pp256 229 t/s and tg32 23.5 t/s; with MTP enabled, generation hits 25.6 t/s — a 17% speedup, with 55/63 drafts accepted.
More from Infra
- First RSI on a Frontier Pre-training Run: Open 801-30B Beats Qwen3-30B — dair_ai · 2026-11-12
- Report: Jensen Huang ousted as Nvidia CEO, sparking founder-leadership debate — PTrubey · 2026-11-12
- Report: Jensen Huang out at NVIDIA, tech world loses legendary founder-mode leader — beffjezos · 2026-11-12
- Global AI compute additions double every 9.1 months, topping 6.3M H100e per quarter: Epoch — dl_weekly · 2026-11-12
- SGLang ported to IBM Spyre in under a week via AI coding agents, 3.6x throughput — PyTorch · 2026-11-12
- Private cloud is a correction to cloud-first thinking as AI drives up infra costs — DavidLinthicum · 2026-11-12