mini-AGI: continual learning model trained from scratch on an 8GB VRAM laptop GPU
udmrzn · x · 2026-09-23
Developer Alexey Borsky open-sourced mini-AGI, a byte-level continual learning language model trained from scratch on a single 8GB RTX 3070 laptop GPU, with most of the code written, debugged and verified by Claude Opus 5. Unlike standard pretrained models with frozen weights, it keeps learning from everything it reads—ingesting 318M characters so far and growing its expert pool to 169 experts. Weights live as ordinary disk files paged into VRAM on demand, so model size is bounded by disk space rather than VRAM, with new capacity grown and unused experts pruned during training. To fight catastrophic forgetting, the backbone learning rate is set to 1/10 of the experts', cutting forgetting from 2.23 nats to 0.0067 nats. The author stresses it's a toy-scale experiment proving continual learning without catastrophic forgetting is feasible on modest hardware.
Related event: Dev trains continually-learning mini-AGI from scratch on an 8GB laptop(4 posts)→
More from Research
- ArcGrav: a cross-model Agent Control Language to cut token overhead — ArchitecturalGravity · 2026-11-12
- Open-source dual-arm robot solves tangram puzzles with 90 demos, ~60% success — DanGrover · 2026-11-12
- FutureHouse's deslopper trains Kosmos to write indistinguishably from human experts — proteinrosh · 2026-11-12
- Fleuret amplifies debate: AI can't slash wet-lab experiments in unmapped biology — francoisfleuret · 2026-11-12
- Recursive's Eureka Machine ships Flint: a sparse 40B model with 2.5B active for local agentic coding — agihouse_org · 2026-11-12
- Recursive bets on open-endedness: agents running massive parallel experiments like scientist communities — agihouse_org · 2026-11-12