Experiment: a learned memory table transfers across model sizes with a small adapter
sn2006gy · reddit · 2026-09-27
The author tested an 'Engram' — an external memory table of associations a model can look up instead of storing facts in weights. A table learned by three small models on synthetic key-to-symbol tasks was attached to two differently sized models of the same family. Mixed results: the exported table was byte-identical across sizes; used directly it matched the no-memory baseline (3.125% accuracy); after training a small adapter per recipient, accuracy reached 71.7% on one held-out wording but stayed at 3.125% on another, with one of six adapter runs missing the pre-set 50% threshold. Evidence that memory tables can move between model sizes and become useful with local tuning — but far from general-purpose transfer. Code and protocol are open-sourced on GitHub.
More from Research
- The brain is a predictive machine: remove reality's correction signal and it hallucinates — alfcnz · 2026-09-27
- Colosseum paper on auditing collusion in multi-agent systems accepted at NeurIPS 2026 — niloofar_mire · 2026-09-27
- MIT talk explains LLMs from first principles, no transformers needed — vishalmisra · 2026-09-27
- Microsoft's ProgramDistill turns interactive web apps into verifiable SWE training tasks — _akhaliq · 2026-09-27
- MIT's pseudorandom codes survey maps the crypto primitive powering AI content watermarks — matthew_d_green · 2026-09-27
- Benchmarking 'Decision Models': Scoring Options Is 7-54x Faster Than Generating Probabilities — vykthur · 2026-09-27