Colibri: Pure C engine runs 2.8T parameter MoE models on consumer hardware
tom_doerr · x · 2026-08-20
Colibri is a zero-dependency inference engine written in pure C that enables running frontier MoE models on consumer and heterogeneous hardware by treating storage, RAM, and VRAM as a single unified memory hierarchy (AI memory multitiering).
It currently supports five model families: GLM-5.2 (744B), Inkling (975B), Kimi K3 (2.8T), DeepSeek V4 Flash (284B), and OLMoE (7B). The project achieves "Tiny engine, immense model" by streaming experts from disk.
More from Infra
- Data Centers Use 627M Gallons Daily, Far Less Than Cattle or Power Plants — rohanpaul_ai · 2026-08-20
- NVIDIA Releases CUDA-Q Algorithms, Open-Source Primitives for Fault-Tolerant Quantum Computing — tomaszbednarz · 2026-08-20
- Dolphin Network launches P2P inference network for idle GPUs — toptickcrypto · 2026-08-20
- Epoch estimate: OpenAI spends roughly 1.2–2.4% of compute on safety monitoring — sjgadler · 2026-08-20
- Hot take: Cerebras inference could speed up OpenAI's R&D loop 20x — Scobleizer · 2026-08-20
- AI Agents Are Not Microservices: The Need for Durable Execution — rseroter · 2026-08-20