Reverse-engineered Splash format ports Qwen3.8-27B to Mac, 88 tok/s at 64k on M5 Max

SeveralViolins · reddit · 2026-09-22

• A developer ported Swift-Qwen3.8-27B as a 4bit model for the Splash inference engine (macOS only). Splash documents the package shape but ships no converter, so the byte-level layout (section offsets, packing order, quantization rule) had to be reverse-engineered from the engine source and verified against the single reference package. Qwen 27B did the heavy lifting, with Opus tidying and spot-checking.
• Performance: 79–129 tok/s decode on M5 Max 128GB; at 64k context, 87.8 tok/s vs 36.6 tok/s for the same weights in oMLX (oQ4e-mtp, mixed 4/5-bit), with the gap widening at longer context.
• Gotchas: Splash's /v1 defaults to greedy decoding, but Swift wants temp 1.0 / topk 20 / topp 0.95, and the format has nowhere to carry generationconfig. — pass them explicitly (write a shim if using DSH).
• Licensing follows upstream projects; Swift Open License v1.0 is free commercially under $1M revenue, and that condition travels with the weights.

Original post →

More from Infra

• Gas-for-AI-datacenters forum co-located with CoRL shows energy demand spillover — carlosdponx · 2026-11-12
• a16z podcast: Mike Volpi on Cisco's 70+ acquisition playbook and the AI buildout — a16z · 2026-11-12
• AMD to pass Intel in server revenue by mid-2027 as share gains 6 points a year — BenBajarin · 2026-11-12
• Study: Reflective Data Center Roofs Actually Cool Daytime Ground by 1.8-2.2°C — SumitGup · 2026-11-12
• Tsinghua's BaGuaLu-2 tames MoE expert-load spikes at runtime, accepted to SC26 — jiqizhixin · 2026-11-12
• antirez backs low-bit DeepSeek 4.1 Flash over Qwen 27B dense on Apple M5 Macs — antirez · 2026-11-12

Infra channel →