LiteRT-LM runs up to 3.5× faster than llama.cpp on Intel Arc iGPU
hi-brawlstars · reddit · 2026-07-28
A Reddit user benchmarked Google’s LiteRT-LM against llama.cpp on an Intel Arc iGPU without matrix cores, using Gemma-4 E2B.
Key results
- Prompt prefill / TTFT: LiteRT-LM is consistently faster, reaching up to 3.5× the throughput of llama.cpp.
- At 32k tokens, first-token latency drops from 210 s in llama.cpp to 80 s in LiteRT-LM, saving about 2.1 minutes.
- Decode speed: llama.cpp with MTP is still ahead at about 30 tok/s, while LiteRT-LM lands at 20–23 tok/s depending on speculative decoding.
Repro details
- Hardware: Intel Core Ultra 7 155U, Intel Arc iGPU, 16 GB LPDDR5x, Windows 11
- Model format: Q4KM GGUF vs auto-int4 .litertlm
- The post includes exact benchmark commands for both stacks.
The main takeaway is that LiteRT-LM appears to be a strong option for reducing prompt-processing latency on consumer Intel iGPUs, especially for long-context workloads.
More from Infra
- fmgo: call Apple's on-device Foundation Models from Go with no CGO and no Swift — Super_Run_8466 · 2026-09-23
- Huawei unveils Peerium architecture: nested BSP unifies million processors into one computer — Dr_Singularity · 2026-09-23
- Grok explains why DeepSeek picked DualPipe + ZeRO-1 over ZeRO-3 on 2048 H800s — TheZachMueller · 2026-09-23
- AI costs fall 47% per quarter, 4x faster than DNA sequencing: Epoch AI — daveholtz · 2026-09-23
- M5 Ultra LLM test: 4x faster prompt processing, but double the power draw — DigitalguyCH · 2026-09-23
- $500 of Dell OptiPlexes become a diskless netboot lab where AI agents can't brick the hardware — colinmcnamara · 2026-09-23