llama.cpp NVFP4 Dot Product Lookup Optimization Boosts ARM Speed by ~5x
pmttyji · reddit · 2026-07-06
A llama.cpp community PR extends UE4M3 lookup table optimization to NVFP4 dot product operations on ARM, reusing existing lookup infrastructure to align ARM implementation with x86. Benchmarks show that the Qwen3.5-4B-NVFP4 model on 4 threads saw pp512 performance jump from 1.89 t/s to 9.97 t/s, achieving a roughly 5x speedup.
More from Infra
- Inference providers built the market squeezing their margins; software is the only way out — AccBalanced · 2026-10-27
- EMNLP Oral: training-free Nexus Sampling cuts KV-cache memory 10x; OpenResearcher tops BrowseComp-Plus at 54.8% — yuz9yuz · 2026-10-27
- Firms offer neoclouds restrictive short-term GPU loans, eyeing cheap GPU pickups — AccBalanced · 2026-10-27
- Anthropic's S-1 exposes the compute financing gap: multi-year commitments vs shifting demand — AccBalanced · 2026-10-27
- Meta unveils 102.4T liquid-cooled network rack and 2-tier Flattened Planar Fabric for AI at OCP 2026 — Meta_Engineers · 2026-10-27
- huggingface_hub 2.3.0 cuts 1M-file listings to 20s, adds live Bucket following — lhoestq · 2026-10-27