Pure-C CPU LLM engine hits 36 tok/s on Xeon, 1.8× faster than bitnet.cpp
shifu_legend · reddit · 2026-07-23
From-scratch pure-C LLM inference engine beats bitnet.cpp by 1.8× on Xeon
Project Zero is a CPU-only LLM inference engine written in pure C99 with zero external dependencies. The author says it outperforms bitnet.cpp by 1.8× on the same Xeon box, reaching 36.25 tok/s vs 19.33 tok/s on an Intel Xeon Emerald Rapids 4C system.
What it supports
- Microsoft BitNet b1.58-2B-4T with ternary weights and a full REPL/agentic loop
- Qwen Bonsai-27B via direct GGUF loading and zero-copy mmap, with a claim that it avoids OOMs on constrained RAM
Why it is faster
- BitNet weights are packed ternary values
- The kernel avoids unpacking to float and instead uses a 3-instruction VBMI path feeding directly into INT8 VNNI accumulation
- Threading uses C11 atomics with a spin-then-sleep pool to avoid futex syscalls
Benchmark call for help
The author only has data from two machines and is asking the community for x86 CPU benchmark results, especially on older AVX2 chips and high-core Xeon/EPYC systems, for both BitNet and Bonsai-27B.
More from coding & agent
- Gary Marcus says Claude Code uses 50+ tools and looks closer to a neuro-symbolic system — GaryMarcus · 2026-07-23
- Seven questions teams should answer before greenlighting an AI agent pilot — kashifmanzoor · 2026-07-23
- A proxy gateway approach to stopping agent tool-call loops — bulleykebaal · 2026-07-23
- Gemini 3.5 Flash-Lite is 71x Cheaper Than Claude for Doc Extraction — rseroter · 2026-07-23
- Cohere to Host Talk on LLM Agent Reliability & Uncertainty Signals — Cohere_Labs · 2026-07-23
- LangChain and Cognition will host a meetup on open memory for agents — LangChain · 2026-07-23