Train 27B models on a 24GB GPU: ternary optimizer cuts memory to 1 byte/weight
oguzeldereli · reddit · 2026-11-11
An independent researcher released TTF (two-timescale threshold flips), a training method that drops AdamW's 16 bytes-per-weight optimizer state down to 1 byte or less, targeting ternary {-1,0,1} networks (BitNet showed ternary matches full precision at scale, but keeps full-precision weights during training).
- A single 24GB RTX 3090 can train a 27B model; a 23B model fits in 16GB; trained weights are 1.6 bits/weight (6GB at 27B), 10x smaller than 16-bit and 5x smaller than Q8;
- A converted, further-trained Qwen3.8 27B scores GSM8K 0.895 (matching full precision), HumanEval 0.84 vs 0.93, running 23 tok/s on a 12GB laptop GPU (improvable with custom kernels);
- Converted Qwen Flash Next is 40.4GB including an 8GB n-gram table — matches benchmarks but forgets facts, works OK with web search;
- Limitations: occasional looping and weak factual memory, but unlike unsloth GGUFs these keep training: fix looping, teach facts, specialize;
- Non-commercial open code, with paper and repos (ttf-ternary / ttf-conversion / ttf-home) plus Hugging Face GGUFs.
More from Infra
- CoreWeave: 2027 GPU renewals could add ~$4B annualized revenue, analyst skeptical — zephyr_z9 · 2026-11-11
- Mark Says George Hotz Was Prescient About Nvidia's Situation — mark_k · 2026-11-11
- Amazon Prime 360 uses AI to rebuild NBA plays in 3D for live, any-angle replays — import_jmr · 2026-11-11
- Altman says OpenAI aims to replace Nvidia in the medium to long term — zephyr_z9 · 2026-11-11
- Chinese University Bought Nvidia H200 Chips Despite US Blacklist — pstAsiatech · 2026-11-11
- Baidu's ERNIE Evolve tops public bests on NanoGPT Speedrun, brings RSI to real inference optimization — xiaohu · 2026-11-11