JITEngine Synthesizes Custom LLM Inference Engines for Macs, Claiming Up to 2.7x Speedup
CShorten30 · x · 2026-11-05
- JITEngine: a new project that "just-in-time" synthesizes an LLM inference engine customized for a specific model + Mac combination, built with SkyDiscover for Apple Silicon — claiming up to 2.7x faster than the fastest alternatives tested.
- Local demo: someone ran Qwen3.8-Flash-Next locally on a 48 GB M4 Pro Mac and had it build Berkeley's Campanile in Minecraft.
- The core idea: skip generic inference engines and generate a bespoke engine per model-hardware pair to squeeze more out of Apple Silicon.
More from Infra
- Samsung's second Taylor fab reportedly fully booked before 2030, mostly for AMD and Nvidia HBM base dies — Beth_Kindig · 2026-11-05
- Why Tsugami's 31% operating margin shows manufacturers can't leave China — PAstynome · 2026-11-05
- Short RL phase on GLM 5.3 Flash costs $372 vs $1,894 for notes, breaks even in 8.7 days — teortaxesTex · 2026-11-05
- Texas lawmaker wants data centers to pay their own energy, recycle water, get community consent — Scobleizer · 2026-11-05
- Memory pricing is squeezing phone chips from both ends: ARM royalty growth cools to 8%, Qualcomm handsets down 26% — tengyanAI · 2026-11-05
- Nationwide voters rejected data centers in 40+ provisions as AI's scientific wins go unnoticed, argues investor — beffjezos · 2026-11-05