JITEngine synthesizes LLM inference engines for Macs, claims up to 2.7x speedup
abeirami · x · 2026-11-05
JITEngine uses SkyDiscover to synthesize custom LLM inference engines just-in-time, specialized per model and Mac configuration.
- 2.72× faster than slotstream for Qwen3.8-Flash-Next on a 48 GB M4 Pro
- 1.36× faster than llama.cpp with MTP for Qwen3.6 on a 24 GB M2
- 1.22× faster than rec for Flint at 252K context on a 48 GB M4 Pro
Users pick their machine, memory budget, and model to get a one-line zsh command for deployment.
More from Infra
- Open-source TrnRT engine serves GLM-5.3-Flash on one Trainium2 box at 3.57M tokens/min — ycombinator · 2026-11-05
- Stanford study: NVIDIA B200 delivers 4x the dollar value per unit of energy vs AMD MI350X — rohanpaul_ai · 2026-11-05
- My Local AI opens Android closed test to run local models free on phones — TheOyinbooke · 2026-11-05
- Amazon CEO: Graviton and Trainium revenue commitments up more than 20x YoY — Beth_Kindig · 2026-11-05
- Semiconductor CEOs: ex-US demand more than sufficient if US buildout slows — inductionheads · 2026-11-05
- ZeroGR paper: filtering-first decoding speeds generative recommendation up to 9.9x — _reachsumit · 2026-11-05