Open-source Swift engine runs Gemma 4 26B on M-series Macs with about 2 GB of RAM

gitpusher42 · hn · 2026-07-29

A developer built TurboFieldfare, an open-source inference engine in Swift + Metal that can run Gemma 4 26B-A4B-IT on any M-series Mac using roughly 2 GB of RAM.

How it works

The model’s 4-bit weights are about 14 GB, which normally makes it hard to run on 8 GB or even 16 GB Macs once the OS and KV cache are included.
The runtime keeps the shared layers and KV cache in RAM.
It streams only the routed experts needed for each token from SSD.
A small expert cache and bounded parallel pread hide some of the SSD latency while the GPU computes the shared layer.

Performance and features

Reported throughput: 5–6 tok/s on an 8 GB M2 MacBook Air.
Reported throughput: 31–35 tok/s on an M5 MacBook Pro.
Includes an OpenAI-compatible local server with streaming, tool calls, and KV-cache reuse for repeated prompt prefixes.
First run downloads about 15 GB of weights from Hugging Face.

The author says they ran 100+ experiments to get this working and invites feedback.

Original post →

Open-source Swift engine runs Gemma 4 26B on M-series Macs with about 2 GB of RAM

More from Infra