Perplexity Open-Sources Lily, a Qwen Inference Engine Built for Apple Silicon

Perplexity announced in a tweet thread on September 3 the open-sourcing of Lily, a local inference engine built for the hybrid compute architecture of its Perplexity Computer. The code (pplx-garden/lily, implemented in Rust) is now live on GitHub. Lily is purpose-built for Qwen3.6-35B-A3B on Apple Silicon (MLX affine 4-bit weights), and a repost mentions a 35% boost in decoding speed.

Confirmed

Lily is officially open-sourced by Perplexity, positioned as the local inference engine within the Perplexity Computer hybrid compute architecture; the GitHub repo is pplx-garden/lily, implemented in Rust.
The optimization target is Qwen3.6-35B-A3B on Apple chips, using MLX affine 4-bit weights.
The design philosophy treats Apple Silicon as a standalone inference platform, mapping Qwen's operators directly onto its compute and memory architecture.
A repost (@inductionheads) relayed the 35% decoding speed improvement and noted it avoids on-device compute bottlenecks.

Why it matters

Perplexity explained in the thread: hybrid compute splits tasks between cloud models and a local model on the Mac, and local inference must keep pace with the overall task rather than slowing it down—hence the need for a dedicated engine.
Perplexity contrasted it with general-purpose frameworks: MLX-LM is a general framework, while Lily is tailor-made for this specific inference workload—illustrating the engineering tradeoff of "dedicated engine vs. general framework," which is a useful reference for on-device LLM deployment.

2026-09-03 ~ 2026-09-03 · 5 related posts

Primary sources

Perplexity open-sources Lily, a Rust/Metal inference server mapping Qwen ops to Apple silicon — perplexity_ai · 2026-09-03 04:05 (UTC+08:00)
Perplexity open-sources Lily, a local inference engine for Qwen3.6 on Apple silicon — perplexity_ai · 2026-09-03 04:04 (UTC+08:00)

[source] Perplexity open-sources Lily, a local inference engine for Qwen3.6 on Apple silicon — perplexity_ai · 2026-09-03
Perplexity details why hybrid compute needs a purpose-built local engine like Lily — perplexity_ai · 2026-09-03
Perplexity explains Lily: purpose-built inference vs general-purpose MLX-LM framework — perplexity_ai · 2026-09-03
[source] Perplexity open-sources Lily, a Rust/Metal inference server mapping Qwen ops to Apple silicon — perplexity_ai · 2026-09-03
Perplexity open-sources Lily, a Metal inference engine for Qwen on Apple Silicon, 35% faster decode than MLX-LM — inductionheads · 2026-09-03