wllama v4.0.0: llama.cpp in the browser, up to 2.8x faster with WebGPU hybrid

ngxson · x · 2026-11-23

wllama, a WebAssembly binding for llama.cpp (1.4k stars), shipped v4.0.0: up to 2.8x faster prefill and loading and 2.5x faster generation on Chrome, no more 4GB size limit (a 5.15GB Gemma 4 E4B now runs at 56 t/s prefill), hybrid CPU + WebGPU inference on Chrome and Firefox for bigger models, plus video input, full-duplex realtime audio, LoRA adapters and speculative decoding (MTP, DFlash, Eagle3). MIT-licensed with an online demo.

Related event: wllama v4.0.0 Boosts In-Browser LLM Inference Up to 2.8x(4 posts)→

Original post →

More from Infra

Infra channel →