Perplexity Open-Sources Lily, a Local Inference Engine for Apple Silicon
On September 3, Perplexity announced in a thread that it open-sourced Lily, a local inference engine built for the hybrid compute architecture of its Perplexity Computer. The code is now on GitHub (pplx-garden/lily, written in Rust), and CEO Arav Srinivas shared the announcement.
Confirmed
- Lily is officially open-sourced by Perplexity, positioned as the local inference engine within Perplexity Computer's hybrid compute architecture; the repo is pplx-garden/lily, implemented in Rust.
- It is optimized for Qwen3.6-35B-A3B on Apple Silicon, using MLX affine 4-bit weights.
- Its design treats the Apple chip as a standalone inference platform, mapping Qwen operators directly onto its compute and memory architecture.
- CEO Arav Srinivas described Lily as a minimal Metal inference server for a single checkpoint.
- A repost by @inductionheads cited a 35% decoding speedup, saying it keeps on-device compute from bottlenecking Computer tasks.
Why it matters
- Perplexity explained that hybrid compute splits tasks between cloud models and local models on the Mac, so local inference must keep pace with the overall task—hence a dedicated engine.
- Perplexity contrasted Lily with general-purpose frameworks: MLX-LM is general, while Lily is purpose-built for this specific inference workload, illustrating the trade-off between specialized engines and general frameworks, relevant to on-device LLM deployment.
2026-09-03 ~ 2026-09-03 · 8 related posts
Primary sources
- Perplexity open-sources Lily, a local inference engine for Qwen3.6 on Apple silicon — perplexity_ai ·
- Perplexity open-sources Lily, a minimal Metal inference server for Qwen3.6-35B-A3B on M5 Macs — AravSrinivas ·
- Perplexity open-sources Lily, a Rust/Metal inference server mapping Qwen ops to Apple silicon — perplexity_ai ·
- [source] Perplexity open-sources Lily, a local inference engine for Qwen3.6 on Apple silicon — perplexity_ai · 2026-09-03
- Perplexity details why hybrid compute needs a purpose-built local engine like Lily — perplexity_ai · 2026-09-03
- Perplexity explains Lily: purpose-built inference vs general-purpose MLX-LM framework — perplexity_ai · 2026-09-03
- [source] Perplexity open-sources Lily, a Rust/Metal inference server mapping Qwen ops to Apple silicon — perplexity_ai · 2026-09-03
- Perplexity open-sources Lily, a Metal inference engine for Qwen on Apple Silicon, 35% faster decode than MLX-LM — inductionheads · 2026-09-03
- [source] Perplexity open-sources Lily, a minimal Metal inference server for Qwen3.6-35B-A3B on M5 Macs — AravSrinivas · 2026-09-03
- Perplexity open-sources its Mac inference server optimized for Qwen 3.6 on Apple Silicon — Specter_Origin · 2026-09-03
1 near-duplicate retellings: inductionheads