A C++20 CPU-native strict-W1 Transformer training runtime seeks feedback
Wonderful-Income7415 · reddit · 2026-07-21
## A CPU-native strict-W1 Transformer training runtime is being built in C++20 The author describes GhostBit, an experimental CPU-native C++20 runtime for training small Transformer language models with strictly binary internal weights in `{-1, +1}`. ### What it includes - strict-W1 BitLinear layers - CPU-native training and inference - early W1/A8 and W1/A1 paths - AVX2-oriented kernels - correctness tests and gradient checks - strict binary checkpoint export verification - reproducible benchmarks - gated A/B experiment decisions (`PROMOTE`, `HOLD`, `REJECT`) - cache, tiling, and compressed optimizer-state experiments ### Current status The author says the project is **not production-ready**, has only been tested on small models and controlled workloads, and does **not** claim CPUs outperform GPUs for general LLM training. They report some encouraging synthetic results but emphasize the need for broader baselines, larger models, more hardware configurations, and independent reproduction. The repository includes more than 100 documented experimental cycles, including failed and inconclusive runs. ### What they want The post asks for honest feedback on whether strict-W1 CPU-native training is a worthwhile research direction, what baselines would be most credible, and whether the work could become a focused paper or reproducible systems contribution. The author also asks about possible research groups or open-source organizations that might be interested.
More from Infra
- More open models and llama.cpp updates are coming, says Merve Noyan — mervenoyann · 2026-07-21
- Why adding a second LLM provider breaks more than the API surface — Ok_Extension6373 · 2026-07-21
- UK AI datacentres face backlash over heat, noise and land use — nordicinst · 2026-07-21
- Fluidstack raises $830M at $7.5B valuation as Anthropic backs a $50B compute buildout — rohanpaul_ai · 2026-07-21
- Early Krea2 Gradio WebUI targets 6GB low-VRAM local runs — Fluid_Kaleidoscope17 · 2026-07-21
- Z.AI starts running a 1GW AI data center built entirely on domestic chips — Polymarket · 2026-07-21