Running LLMs Purely via WASM in the Browser
celsowm · reddit · 2026-07-15
This project turns LiquidAI's LFM 2.5-350M into an inference demo that runs entirely locally in the browser: no backend, no WebGPU dependency, executing directly via pure WebAssembly.
Key Implementations
- Hand-written WASM SIMD kernels covering gemm/gemv, GQA attention, conv, RoPE, etc.
- Model weights are 4-bit quantized (Q40), with attention projections using Q8.
- Supports batched prefill to avoid token-by-token prompt preprocessing.
- The frontend provides a chat UI, streaming output, a stop button, real-time tok/s, and markdown rendering.
Deployment
- Weights are hosted on Hugging Face: celsowm/lfm2.5-350m-wasm-q40
- The app is statically deployed on GitHub Pages.
- Upon page load, it fetches vocab/merges from Pages and weights from HF, running all inference locally.
The author also included links for the live demo, GitHub repository, and model weights.
More from Infra
- SF Compute founder: buying compute is 'an absolutely awful experience' right now — IgorCarron · 2026-09-11
- SmolVM open-sources persistent computer infrastructure for agents that outlive chat sessions — aniketmaurya · 2026-09-11
- PyTorch Day Korea 2026 launches first offline conf, CFP closes Sept 13 — PyTorch · 2026-09-11
- Local LLM server dilemma: 4x CMP-170HX (price up 53% in 20 days) vs Mac Studio M5 Ultra — rumboll · 2026-09-11
- llama.cpp lands Flash Attention tuning for RDNA4, big prefill gains on AMD — pmttyji · 2026-09-11
- Your p99 latency benchmark may be lying: a deep dive into coordinated omission — Franc0Fernand0 · 2026-09-11