Frozen Qwen2.5-7B answers from a 4.7 MB memory file, no retraining, open code
Nearby_Indication474 · reddit · 2026-10-10
A developer released AkbasCore MAM-BÇ, an open-source experiment: a frozen Qwen2.5-7B-Instruct (zero weight changes) answers questions from numerical memory that was saved as a 4.7 MB file, reloaded bit-for-bit, and plugged back in — the source text is never shown to the model. Code, raw logs, and a DOI are public.
How it works (a third path beyond fine-tuning and RAG):
- Write: each fact passes once through the frozen model; they keep the layer-3 residual stream as a retrieval key plus attention keys/values of layers 0–3 (a "cartridge" containing tensors, no text)
- Select (FRAM): the question is encoded in the same frame; cartridges are scored via a log-sum-exp soft-maximum kernel; hyperparameters were fixed on only 12 calibration questions, with 12 holdout questions untouched
- Attach (CUT3): the top 5 cartridges merge; lower-layer keys are exactly rotated with the model's own rotary embeddings, upper layers computed only for new rows, earlier rows bit-identical
- Save/reload: the merged 28-layer KV memory (57,344 bytes/token) is written to disk, read back, verified bit-for-bit, and reattached
The author claims the model correctly cites a bank of 96 facts purely from stored internal states.
More from Research
- EMNLP paper: diffusion LMs internally encode latent denoising timestep representations — gsarti_ · 2026-10-23
- KU Leuven and VIB open two Group Leader professorships in AI for Biology, start 2027 — jpillowtime · 2026-10-23
- Subspace Tuning curbs LLM catastrophic forgetting by freezing principal singular directions — scottniekum · 2026-10-23
- Perceptic launches PercepticOS, an AI workspace for 'scientific superintelligence' — nathanbenaich · 2026-10-23
- EPFL's Bosselut to talk long context as test-time learning at ELLIS seminar — andre_t_martins · 2026-10-23
- DRATS: Adaptive Task Sampling Boosts Worst-Task Performance in Multi-Task RL — Cohere · 2026-10-23