JITEngine synthesizes custom LLM inference engines for Apple Silicon, up to 2.7x faster
CShorten30 · x · 2026-11-12
mertcemri introduced JITEngine: built with SkyDiscover, it can synthesize an LLM inference engine customized for your specific model and Mac just in time, claiming up to 2.7x faster than the fastest alternatives tested, targeting Apple Silicon.
- The idea is JIT: instead of running every model on a general-purpose engine, generate an engine tailored to the model-hardware pair
- Currently for Mac/Apple Silicon; commenters hope for future Linux cloud GPU node support
- Shared as a thread teaser, with details to follow
More from Infra
- Homemade inference box pairs AMD CPUs with Huawei Ascend 950PR NPUs — AIFlow_ML · 2026-11-12
- Jensen Huang leaving isn't the problem — AI's four-layer single points of failure are — krishnan · 2026-11-12
- Studies: Data Centers Cause Just 4% of US Home Rate Hikes as Senator Backs Moratorium — Eliv_nurotic · 2026-11-12
- vLLM ships Day-0 support for MiniMax-M3.1-Flash with 10.2x throughput on Rubin NVL72 — vllm_project · 2026-11-12
- vLLM benchmarks MiniMax-M3.1-Flash on Vera Rubin NVL72: 10.2x per-chip throughput vs GB300 — vllm_project · 2026-11-12
- Meta FAIR open-sources e-means: 10–45x faster k-means with lower loss than Lloyd's algorithm — generativist · 2026-11-12