ijah: a new engine for running Qwen 3.8 Flash-Next locally on M5 Max Macs
SeveralViolins · reddit · 2026-10-20
After Splish (a speed-focused fork of Splash for M5 Max), the author released ijah, borrowing from the EXL3 engine Sushi with new kernels, aiming for optimal quality and speed running Qwen 3.8 Flash-Next / Swift 1.5 Flash Next on 128GB M5+ Macs. Flash-Next had underwhelmed them before—Q4 margins on the 27B never justified the RAM footprint—but Sushi's EXL3 trellis weights made 5b/6-8b models cook and flipped their opinion. On 5b-6/8b models, ijah's quality is 4-5× closer to bf16 than Jundot's oQ4e (KLD 0.033/0.009/0.015 vs 0.128/0.036/0.078) and 2× closer than Sushi-4bpw. Decode is 100 tok/s (90-110 via server), 6% behind oMLX but with 5-bit experts; with the DFlash2 drafter it hits 120 tok/s (pending verification). Open source on GitHub, with a ko-fi link.
More from Infra
- REAX inference engine runs Perplexity Decider up to 6x faster, free for individuals — airesearch12 · 2026-10-20
- BitNet CPU inference is silently wrong: perplexity 6x reference, one-line fix works — matheus-c-feitosa · 2026-10-20
- Data centers are the real paperclips and the global bond market is the steel — m2saxon · 2026-10-20
- AMD: World now consumes 40 quadrillion inference tokens monthly, up 95x in two years — Beth_Kindig · 2026-10-20
- Meta details Prometheus gigawatt-scale supercluster, targeting ~1M GPUs — BenBajarin · 2026-10-20
- Cloudflare launches Clairic, a System One extraction model with single-pass, token-free typed outputs — Cloudflare Blog · 2026-10-20