Running LLMs Purely via WASM in the Browser
celsowm · reddit · 2026-07-15
This project turns LiquidAI's LFM 2.5-350M into an inference demo that runs entirely locally in the browser: no backend, no WebGPU dependency, executing directly via pure WebAssembly.
Key Implementations
- Hand-written WASM SIMD kernels covering gemm/gemv, GQA attention, conv, RoPE, etc.
- Model weights are 4-bit quantized (Q40), with attention projections using Q8.
- Supports batched prefill to avoid token-by-token prompt preprocessing.
- The frontend provides a chat UI, streaming output, a stop button, real-time tok/s, and markdown rendering.
Deployment
- Weights are hosted on Hugging Face: celsowm/lfm2.5-350m-wasm-q40
- The app is statically deployed on GitHub Pages.
- Upon page load, it fetches vocab/merges from Pages and weights from HF, running all inference locally.
The author also included links for the live demo, GitHub repository, and model weights.
More from Infra
- Why a 1GW Chinese AI data center may be plausible after all — teortaxesTex · 2026-07-22
- LFM2.5-8B-A1B doubles its tokenizer vocab and cuts on-device decoding time up to 3.7x — maximelabonne · 2026-07-22
- China’s AI arms race is increasingly defined by chips, data centers, and open models — BenBajarin · 2026-07-22
- Agent search bottlenecks are now about variance, not raw latency — rohanpaul_ai · 2026-07-22
- Gavin Baker argues Nvidia may be one of open source AI’s biggest supporters — GavinSBaker · 2026-07-22
- AI Power Demand Exposes US Energy Gap, Urging Shift from Scarcity to Abundance — bradneuberg · 2026-07-22