LFM3 runs fully offline on iPhone: 40 tok/s text, custom ExecuTorch export open-sourced
K4anan · reddit · 2026-10-30
Liquid AI's LFM3-1.7B and LFM3-VL-2B now run fully offline on iPhone and Android via the open-source Private Mind app. LFM3-VL-2B correctly reads a Japanese ramen menu — skipping the extra-spicy bowl and spotting a "cash only" note.
Benchmarks on iPhone 16 Pro: LFM3-1.7B text (MLX 4-bit on GPU) 40 tok/s with first token under 0.5s; LFM3-VL-2B with a photo 9s to first token, 20 tok/s decode.
The author rewrote the decoder for ExecuTorch export: LFM3 uses 24 layers mixing gated short convolutions (width 7), full and 256-window sliding attention, per-head sigmoid gates, RMSNorm on Q/K, and layer-type-specific RoPE. They worked around ExecuTorch's ring-buffer cache (2x window, capping forwards at 512 tokens) with a window+2048 buffer plus attention masking — single calls now take 2048 tokens with 8192 context. Max logit deviation vs reference: 0.00005. Exports are on Hugging Face.
More from Infra
- Running a French LLM on French Kubernetes in French datacenters — RemiCadene · 2026-10-30
- Starlink Connects 250 Remote Zambia Clinics, Enabling EHR and HIV Care Access — XFreeze · 2026-10-30
- Intel warns memory crunch will cut TAM 10%; commentator sees PC units down 20% — teortaxesTex · 2026-10-30
- MediaTek's AI ASIC revenue hits $2.5B in Q4, beats guidance; 448G SerDes tapes out — zephyr_z9 · 2026-10-30
- Gartner: 80% of independent neoclouds will be acquired or fold by 2030 — jeremyakahn · 2026-10-30
- Micron stock drops 6% as Intel sees no end to memory shortage — JOBhakdi · 2026-10-30