Dev Trains Continual-Learning Model From Scratch on 8GB VRAM Laptop, Now at 530M Params
Another__one · reddit · 2026-09-21
A developer shared the mini-AGI project on Reddit: a continually learning model trained from scratch on an 8GB VRAM laptop using a batch-1 stream of data, now at 530M parameters and still growing.
Two key ideas:
- Dynamic MoE: experts are added and pruned during training, with only a small subset loaded at any time, so parameter count is bounded only by disk space;
- Batch-1 stream training: no need to store large randomized batches and gradients, drastically cutting VRAM needs.
The motivation is dissatisfaction with being unable to train 1B+ models on consumer hardware; the author wants full control over what the model sees. The model reads the 7.8B-character corpus as one continuous stream in 32K-character passages, "just like you or I would," and is still training. The author says the project was built by brainstorming with Claude and would have been impossible otherwise. It's open source — you can git clone and run it — and includes a promising scaling curve.
Related event: Dev trains continually-learning mini-AGI from scratch on an 8GB laptop(4 posts)→
More from Research
- SGLang Megakernel Pushes Muse Spark to 1,109 tok/s, 2.1x Over Baseline — hsu_byron · 2026-11-12
- ArcGrav: a cross-model Agent Control Language to cut token overhead — ArchitecturalGravity · 2026-11-12
- Open-source dual-arm robot solves tangram puzzles with 90 demos, ~60% success — DanGrover · 2026-11-12
- FutureHouse's deslopper trains Kosmos to write indistinguishably from human experts — proteinrosh · 2026-11-12
- Fleuret amplifies debate: AI can't slash wet-lab experiments in unmapped biology — francoisfleuret · 2026-11-12
- Recursive's Eureka Machine ships Flint: a sparse 40B model with 2.5B active for local agentic coding — agihouse_org · 2026-11-12