Platypus: an open-source continual-learning LM that infers new facts while it "sleeps"
ChrisLLM · reddit · 2026-11-01
Platypus is an open-source (Apache 2.0) byte-level LM built on complementary learning systems (CLS) theory — a fast hippocampus + slow cortex split:
- Learns 10 Pile domains sequentially with no catastrophic forgetting (slightly negative in runs)
- Keeps a fast episodic fact store beside trained weights
- "Sleeps" after each domain: links facts into two- and three-hop conclusions, verifies them against cited sentences, writes them back
- Revises beliefs on changed facts, abstains without evidence; rule-free mode keeps links only when they pay for themselves in compression
Results: 1.00 two-/three-hop recall on planted chains with no added forgetting; natural prose is harder (0.77 two-hop). The core open problem is consolidation — arbitrary facts don't transfer into weights at safe learning rates, as CLS theory predicts. Schema-level replay experiments at 171M parameters are underway.
More from Research
- Open-source memory layer ICE loses to vector-RAG on LongMemEval: 43% vs 69.5% — Deepnar_S · 2026-11-01
- Tiny 109M model beats 72B models at image-text alignment with 42x less data — Lazylion2 · 2026-11-01
- Can you name these three ML architectures from blank diagrams? — cneuralnetwork · 2026-11-01
- TIL: Boolean circuits can be cyclic — NOR-gate latches and Rivest-style loops — thomasahle · 2026-11-01
- The Path of Averaging: Tracing the Line from Attention to TTT — SonglinYang4 · 2026-11-01
- AI Impacts' '5% extinction' framing was misleading: the survey also asked about severe disempowerment — NathanpmYoung · 2026-11-01