Colibri: pure-C inference engine runs 2.8T-parameter MoE models by streaming experts from disk
tom_doerr · x · 2026-10-13
Colibri is a zero-dependency, pure-C open-source inference engine that treats storage, RAM, and VRAM as a single memory hierarchy, streaming MoE expert weights from disk to run frontier models up to 2.8T parameters on consumer hardware.
- Supports nine model families: GLM-5.2/5.3 (744B), GLM-5.3-Flash (321B, with vision), Inkling (975B), Kimi K3 (2.8T), DeepSeek V4 Flash (284B), DeepSeek V4.1 Flash (552B, with vision), Qwen3.8-Flash-Next (125B + 51B n-gram), Qwen3.6, and OLMoE (7B) — one C file each, sharing col chat/serve/web frontends
- Design philosophy: no SLA on speed, but a hard guarantee on semantics — no silently changing model precision or router semantics
- 41.6k GitHub stars, Apache-2.0, positioned as an open research platform for inference-side systems work
More from Infra
- Ex-Apple engineer eyeing Surface with NVIDIA RTX Spark: 128GB + CUDA in a backpack — LinusEkenstam · 2026-10-13
- Local 125B MoE on RTX 5090 finishes Three.js horse challenge in 23 minutes — Beneficial-Ad-8127 · 2026-10-13
- ML Veteran John Langford Endorses GPU MODE's Open Benchmark Competition Model — JohnCLangford · 2026-10-13
- Transcribing an hour of audio in ~13s on a MacBook Air — amrrs · 2026-10-13
- QuantWM: training-free 2-bit KV cache quantization cuts video world model memory 6x without flicker — jiqizhixin · 2026-10-13
- User buys two DGX Spark units at ~$6,100 each, finds local GLM 5.3 setup info scarce — MasterNomie · 2026-10-13