Drex 2.0 runs fully local on a Mac: 21GB weights, 113ms per decision
testingcatalog · x · 2026-11-02
Nace.AI released a 4-bit build of Drex v2.0 (Q4KM, a 21 GB GGUF) that runs entirely on Apple silicon Macs with 36 GB+ memory, via its open-source engine Kapitsa plus forks of llama.cpp and Ollama. Weights are open (Nace.AI Open RAIL-M, modified); Kapitsa is Apache-2.0.
On an M5 Pro (48 GB), median decision time on a short support ticket was 113 ms with Kapitsa, 248 ms with llama.cpp, and 282 ms with Ollama. Follow-up questions on an already-read document take just 48 ms, since Kapitsa keeps document state and only scores the new question. On an M3 Pro (36 GB), llama.cpp takes 441 ms per decision. Nothing leaves the machine; setup, serving, and requests all run from the terminal.
More from Infra
- clusyio's Portal lets Python jobs migrate to a new machine mid-run — AIFlow_ML · 2026-11-02
- Same intelligence gets 10x cheaper every 4-6 months, says Artificial Analysis — altryne · 2026-11-02
- tinybox owner: $11k-$22k local AI boxes beat buying GPUs outright — CedricMakes · 2026-11-02
- OpenAI plans to rebrand data centers as "AI compute campuses" after midterms — TorturedPoet30 · 2026-11-02
- Battery expert Maker_Matthew joins X, shows serialized batteries rolling off the line — yacineMTB · 2026-11-02
- Wan 2.2 color pulse fix and other hard-won tips from rendering a music video on one AMD GPU — wilppuse · 2026-11-02