Pre-training done in 68 hours with 4-bit used all the way from pre- to post-training
teortaxesTex · x · 2026-11-06
Quoting @fengyao1909: the team ran 4-bit precision all the way from pre-training to post-training, finishing pre-training in just 68 hours. If verified, it signals an aggressive low-cost training path with minimal compute requirements.
- Full 4-bit pipeline: pre-training through post-training
- Pre-training completed in 68 hours
- Team describes itself as a continuously shipping "model factory"
More from Infra
- Jane Street shares its experience building with NVIDIA's cuTile language — GuggerSylvain · 2026-11-06
- AI overbuild skeptics miss the demand: agents, cyber AI and 0.1% enterprise penetration — builderjaydub · 2026-11-06
- Against datacenter doom: three reasons the buildout continues — max_paperclips · 2026-11-06
- LSEG cuts 5-year market backtest from hours to ~1 minute; Tampa General adds AI safety gate — NVIDIAAI · 2026-11-06
- TensorFold 1.2.3 ships faster Flash Next: +21% GB10 decode, M3 Ultra 64K-context reply in 8s — HankYeomans · 2026-11-06
- Autonomous Launches Local AI Workstations: 8-GPU Unit at $99,700 with Free 24-Hour Setup — dee_hw · 2026-11-06