LiteRT-LM runs up to 3.5× faster than llama.cpp on Intel Arc iGPU
hi-brawlstars · reddit · 2026-07-28
A Reddit user benchmarked Google’s LiteRT-LM against llama.cpp on an Intel Arc iGPU without matrix cores, using Gemma-4 E2B.
Key results
- Prompt prefill / TTFT: LiteRT-LM is consistently faster, reaching up to 3.5× the throughput of llama.cpp.
- At 32k tokens, first-token latency drops from 210 s in llama.cpp to 80 s in LiteRT-LM, saving about 2.1 minutes.
- Decode speed: llama.cpp with MTP is still ahead at about 30 tok/s, while LiteRT-LM lands at 20–23 tok/s depending on speculative decoding.
Repro details
- Hardware: Intel Core Ultra 7 155U, Intel Arc iGPU, 16 GB LPDDR5x, Windows 11
- Model format: Q4KM GGUF vs auto-int4 .litertlm
- The post includes exact benchmark commands for both stacks.
The main takeaway is that LiteRT-LM appears to be a strong option for reducing prompt-processing latency on consumer Intel iGPUs, especially for long-context workloads.
More from Infra
- LightRAG v1.5.3 adds a one-time Milvus migration and hardens production edge cases — JeremyCMorgan · 2026-07-28
- llama.cpp adds DSpark speculative decoding and asks for speed results — pmttyji · 2026-07-28
- Nvidia puts an open vision-language-action model on Hugging Face — theteknosaur · 2026-07-28
- “Model eats harness” is really about deployment-driven harness evolution — m_wulfmeier · 2026-07-28
- Agentic AI system reportedly cost $1.2M a month before being cut to $100K — emmanuelvivier · 2026-07-28
- Wistron’s Early Nvidia Bet Turned It Into One of AI’s Biggest Supply-Chain Winners — pstAsiatech · 2026-07-28