KDA memory decay uses radioactive-style half-life math, with 69-token lifetime at α=0.99
CatAstro_Piyush · x · 2026-07-29
KDA’s memory decay follows the same half-life math as radioactive decay: if a channel evolves as Sn = α^n S0, then its half-life is n½ = ln(0.5) / ln(α). The post notes that α = 0.99 gives a half-life of about 69 tokens.
The attached formula image shows KDA learning different decay parameters across memory channels, so each channel can operate on its own timescale. In other words, the model is not using one universal memory horizon, but a set of learned ones.
More from Research
- Netflix details its production LLM judge: hundreds of thousands of recommendations scored weekly — omarsar0 · 2026-08-24
- Nature Comment: Provenance, not interpretability, grounds trust in autonomous science — gabepgomes · 2026-08-24
- New Architecture RHEA: Train 1B Model on 8GB VRAM — zemondza · 2026-08-24
- Trained two 16M-param models to do generative CAD with real physics — debreuil · 2026-08-24
- Claude model helps discover complex structure on S^6, solving 60-year-old math problem — Singularitarian · 2026-08-24
- Study: Agents read instructions/notes 60.5% of the time, rarely touch API docs — dair_ai · 2026-08-24