Hierarchos Native cuts Intel HD 520 streaming latency 58.3%, passes full RDNA3 qualification
PhysicsDisastrous462 · reddit · 2026-11-03
Hierarchos Native, a Rust+Vulkan backend for LLM training, fine-tuning and inference requiring no Python/PyTorch/CUDA/ROCm, shipped a major update. Key changes: DeepSeek V4.1 Flash port with fused expert execution, batched expert uploads and native LoRA/rsLoRA training; broader Intel Gen9 coverage (Qwen3-Next padding, DBRX expert layouts, T5/BART/Switch streaming paths); reduced streaming overhead via retained file handles and coalesced reads. On an i5-6200U / Intel HD 520, Llama-wide streaming median forward latency dropped from 11.016ms to 4.597ms (-58.3%), with GPU submissions per forward down from 19 to 4; across eight scaled controls, streaming medians improved 42.4-78.4%. RDNA3 qualification passed 166/166 forward runs and 72/72 generation requests with max CPU logit error below 2e-7 (rtol=0) and bit-identical resident/streamed outputs, plus a fix for a padded GPT-2 numerical parity issue; V4.1 resident decode latency fell 50.2%.
More from Infra
- Mistral Large 4 (1T params) runs locally on two M3 Ultras at 20 tok/s via MLX-VLM — andrejusb · 2026-11-03
- AEHR jumps 16% and LSCC 9% after hours as both chip stocks report earnings — BenBajarin · 2026-11-03
- RidgeFill: training-free prefill skipping cuts TTFT 1.6x, 64K request 82.6s to 47.5s — Public_Umpire_1099 · 2026-11-03
- Stanford physicist argues space compute cooling is feasible with a black-sphere thought experiment — shaunmmaguire · 2026-11-03
- Interactive map of AI debt: 25 parties, 58 obligations, and no real counterparty — W1RELESS · 2026-11-03
- TensorFold 1.0: open-source local inference hits 874 tok/s on RTX PRO 6000 — Jasonio · 2026-11-03