CLM-v0.1-8B ported to MLX: 99% agreement with official vLLM at ~336 tok/s on M3 Pro
WebAssemblyMan · reddit · 2026-09-26
A community developer ported CLM (Contrastive LM, a self-described "System One model") to Apple MLX. Like TypeSafe AI's closed API-only Jev, it skips text generation and returns typed answers — yes/no, choice, score — via embedding similarity.
Key numbers:
- Architecture: frozen Qwen3-8B encoder + tiny fp32 heads, Apache-2.0
- 8-bit checkpoint, 7.5 GiB, 336 tok/s at 9 GB peak on an M3 Pro
- Checked against the authors' vLLM server on 778 questions: 99.0% top-1 agreement, within run-to-run noise
It runs fully local and free with no API calls, ideal for high-volume yes/no / multiple-choice decisions. An unofficial port, not reviewed by the CLM authors; weights and code are on Hugging Face.
More from Infra
- Why this builder quit server racks: fried motherboards and a ~$1,500 housing bill — TheZachMueller · 2026-09-26
- Farmer's photo exposes 62 unpermitted gas generators powering Microsoft AI data center, $1.1M fine — mkheck · 2026-09-26
- LLM routing saved 33.2% vs premium models in 640-request pilot, but a fixed mid-priced model beat it — smakosh · 2026-09-26
- Akamai CEO on $11.6B Anthropic cloud deal: 'this business is going to help our margins' — pdamodaran · 2026-09-26
- MLXUI: Open-Source Local AI Browser for Apple Silicon with One-Click Model Installs — WebAssemblyMan · 2026-09-26
- Samsung now shipping LPDDR5 even in its lowest-end phones amid LPDDR4X shortage — zephyr_z9 · 2026-09-26