Raon-1-mini-preview runs natively on hipfire but 200x slower prefill than llama.cpp, patch released
Queasy_Asparagus69 · reddit · 2026-10-30
A dev ported KRAFTON's Raon-1-mini-preview (28.8B/4.3B MoE) to a hipfire-native MQ4 quant and benchmarked it on a 20 GB 7900 XT (ROCm 7.1): prefill is 200x slower (12.6 vs 2,055 t/s) and decode 10x slower than llama.cpp Q4KM, likely because hipfire handles the recurrent state token-by-token with no batched prefill kernel. The port is 1.29 GB smaller than the official GGUF and validates the full hybrid architecture (RDA linear attention + gated MLA + mHC + 128-expert MoE) against fp32 reference with mean KL 2.06. The repo includes a patch for hipfire v0.4.2.1's 'stream must begin with genstart' failure, plus findings on broken thinking-mode frames, greedy-sampling needle-test failures absent on the official GGUF, and a 65K context ceiling at 20 GB. Posted as a reproducible before-state for future Raon kernel work.
More from Infra
- Cerebras breaks down the math of deploying multi-trillion parameter models at scale — ceciletamura · 2026-10-30
- Crusoe closes round at $31B valuation, from Bitcoin mining to AI data centers — nathanbenaich · 2026-10-30
- AWS hits $184B ARR, up 39.3% year over year — RihardJarc · 2026-10-30
- AI boom: US data center & computer spending now 50% larger than home construction — HaydnBelfield · 2026-10-30
- Whatever Happened to SanDisk's High Bandwidth Flash? 3TB Per GPU Promise Met With Silence — Zombiecidialfreak · 2026-10-30
- Why local inference is inefficient: moving weights costs ~100x more than computing — cis_female · 2026-10-30