Colibri: Run 2.8T-parameter MoE models on a 25GB laptop with pure C inference engine
alex_verem · x · 2026-08-16
Developer alexverem introduces Colibri, an open-source inference engine that runs AI models up to 2.8 trillion parameters on a 12-core laptop with 25GB RAM. The project has 24.8k stars on GitHub.
Colibri exploits the sparse activation of MoE models (e.g., GLM-5.2 has 744B parameters but activates only 40B per token), treating storage, RAM, and VRAM as a unified hierarchy, streaming experts from disk on demand and caching hot ones. It's pure C with zero dependencies and runs without a GPU.
Performance: 5.8 tokens/s on 6x RTX 5090 (744B model), 1 token/s on a laptop GPU, and 0.1 token/s on the original 25GB machine.
More from Infra
- Beating cuBLAS by 4.7%: NVFP4 Kernels Hand-Built on GB300, 100% Claude-Generated — pranjalssh · 2026-08-16
- Qwen 3.8 runs 151% faster on Apple Silicon via community challenge — gajesh · 2026-08-16
- a16z: NeoClouds Repurpose Crypto Infrastructure for AI Compute Boom — a16z · 2026-08-16
- OpenAI Previews GPT-5.6 Sol Ultrafast Mode: 14x Speed Boost Powered by Cerebras — Justgototheeffinmoon · 2026-08-16
- Local AI doesn't need to replace frontier cloud models; hybrid is the destination — ingliguori · 2026-08-16
- Polymarket: 69% Chance a US State Enacts Data Center Moratorium by End of 2026 — Polymarket · 2026-08-16