mini-AGI: Looped Continual-Learning Transformer With Evolutionary Experts Trained on a Laptop
returnity · reddit · 2026-09-23
Reddit user returnity shared mini-AGI, a continual-learning dynamically looped transformer trainable on a laptop, with unusual design choices:
- Loop architecture: dynamic per-token recurrent depth up to 24 cycles
- Self-supervised, tokenizer-free — reads raw bytes directly
- MoE with 8 active, 32 routed experts in VRAM plus smart caching of 96 more; weights paged from SSD on demand
- Dynamic growth: builds new experts and grows parameter counts as needed
- Evolutionary growth trials new experts and prunes unused ones
- Catastrophic forgetting mitigated via slow-trunk/fast-expert learning rate split
Weights will be released "in a couple weeks" once training reaches GPT-2 levels; the trend line has held 15-fold so far but may bend.
Related event: Developer Trains 530M-Parameter Continual Learning Model on 8GB Laptop(4 posts)→
More from Infra
- Analyst: AMZN trades at just AWS + Anthropic stake value, market blind on non-semis AI plays — RihardJarc · 2026-09-23
- ASML Says It Sells Zero Machines in Europe as No Chip Factories Are Built — 2C_ornot2C · 2026-09-23
- Qwen Image 2.1 Fast FP8 Wows Redditors: Premium Images From Under 10GB — 108er · 2026-09-23
- Chutes names new CEO, pretrains 8B MoE in public, hits 15.5k tok/s on one RTX 5090 — markjeffrey · 2026-09-23
- Halo post-training framework claims 2.8x TRL throughput, accused of cherry-picking benchmarks — _ScottCondron · 2026-09-23
- How OpenAI Built GPT-Live: Engineers Deep-Dive with ByteByteGo — juberti · 2026-09-23