BitRun v0.2 runs 8B ternary LLMs on CPU in 2GB RAM with live wattage telemetry
The_VibePreneur · reddit · 2026-10-16
BitRun v0.2.0-beta brings 1-bit LLM inference to everyday CPUs with a new split-screen TUI called Cockpit: tokens stream on the left while the right panel shows live CPU package power in watts, joules per token, P-core/E-core loads, and time-to-first-token.
On the model side, the project detached from Microsoft's frozen submodule, rebased on modern llama.cpp, fixed a GGUF Type 42 collision, and got PrismML's 8B ternary model (bonsai-8b) running entirely in CPU memory in 2GB RAM.
Benchmarks: on an Apple M4 Pro (zero GPU), bonsai-1.7b hits 73 tokens/sec at 11.7W (0.31 J/token); Falcon3 1B also runs 73 t/s; the 8B model does 21 t/s at the same 11.7W. On a Core i5-12600K, bonsai-1.7b generates 85 t/s at 22W and BitNet 70M clocks nearly 600 t/s. bitrun serve exposes a local OpenAI-compatible endpoint on port 8080 for Cursor or Open WebUI. Code and binaries on GitHub.
More from Infra
- Meta's open Muse Spark needs 493GB of memory — far beyond any consumer laptop — TejasKumar_ · 2026-10-16
- CoreWeave plunges 20% as credit markets price >60% default odds; is this dot-com all over again? — kimmonismus · 2026-10-16
- MiniCPM-V 4.7 35B-A3B hits 17+ tokens/s on iPad with ~3GB memory — techNmak · 2026-10-16
- China rolls out 'token factories' and a 2026-28 token economy action plan — nordicinst · 2026-10-16
- Vercel Sandbox Drives go GA: attach TBs of data in sub-second, fork instantly — cramforce · 2026-10-16
- NVIDIA launches RTX Spark for local AI on Windows PCs — ostrisai · 2026-10-16