mini-AGI: Looped Continual-Learning Transformer With Evolutionary Experts Trained on a Laptop
returnity · reddit · 2026-09-23
Reddit user returnity shared mini-AGI, a continual-learning dynamically looped transformer trainable on a laptop, with unusual design choices:
- Loop architecture: dynamic per-token recurrent depth up to 24 cycles
- Self-supervised, tokenizer-free — reads raw bytes directly
- MoE with 8 active, 32 routed experts in VRAM plus smart caching of 96 more; weights paged from SSD on demand
- Dynamic growth: builds new experts and grows parameter counts as needed
- Evolutionary growth trials new experts and prunes unused ones
- Catastrophic forgetting mitigated via slow-trunk/fast-expert learning rate split
Weights will be released "in a couple weeks" once training reaches GPT-2 levels; the trend line has held 15-fold so far but may bend.
Related event: Dev trains continually-learning mini-AGI from scratch on an 8GB laptop(4 posts)→
More from Infra
- Gas-for-AI-datacenters forum co-located with CoRL shows energy demand spillover — carlosdponx · 2026-11-12
- a16z podcast: Mike Volpi on Cisco's 70+ acquisition playbook and the AI buildout — a16z · 2026-11-12
- AMD to pass Intel in server revenue by mid-2027 as share gains 6 points a year — BenBajarin · 2026-11-12
- Study: Reflective Data Center Roofs Actually Cool Daytime Ground by 1.8-2.2°C — SumitGup · 2026-11-12
- Tsinghua's BaGuaLu-2 tames MoE expert-load spikes at runtime, accepted to SC26 — jiqizhixin · 2026-11-12
- antirez backs low-bit DeepSeek 4.1 Flash over Qwen 27B dense on Apple M5 Macs — antirez · 2026-11-12