Single Pure-C99 Inference Engine Runs Both BitNet Ternary and GGUF, No Python or CUDA
shifu_legend · reddit · 2026-09-15
Project Zero is an inference engine written from scratch in pure C99 that runs both BitNet ternary and standard GGUF models from a single binary with only GCC and make — no Python or CUDA. It hits 36 tok/s on BitNet b1.58-2B-4T on a Xeon (1.8x faster than bitnet.cpp) and 100 tok/s on SmolLM2 F16 on an i5-11300H, though llama.cpp still leads Q4K by a wide margin. Ships with an OpenAI-compatible API and SSE streaming; a pre-built x86 Linux binary is available.
More from Infra
- "Pace the frontier"? Investor says panic selling of AI chip stocks precedes biggest ramp ever — firstadopter · 2026-09-15
- Nvidia CMP 170HX Modded From 8GB to 64GB With 1.49 TB/s Bandwidth — _Boffin_ · 2026-09-15
- NVIDIA: full-stack NIM tuning delivers 2.5x more concurrent users on Nemotron 3 Ultra — NVIDIAAI · 2026-09-15
- How Much Does Local LLM Inference Really Cost? A Dev Added an Electricity Calculator — giveen · 2026-09-15
- Hugging Face Rounds Up Which Open LLMs Are Best for On-Device Inference — NielsRogge · 2026-09-15
- Dev weighs ChatGPT subscription via OAuth vs API pricing for a production RAG app — builtforoutput · 2026-09-15