Injecting hidden-state memory cartridges into frozen Mistral-7B and Qwen2.5-7B, weights untouched
Nearby_Indication474 · reddit · 2026-10-16
Second experimental implementation of AKBASCORE Silicon Hippocampus v1.4, now on frozen Mistral-7B-Instruct-v0.3 after the first on Qwen2.5-7B-Instruct. Question: can independently extracted internal representations be stored as numerical memory cartridges, assembled incrementally, and reinjected at layer boundaries with weights frozen?
Unlike ordinary KV-cache reuse, each fact is processed alone; a hidden-state checkpoint plus lower-layer K/V tensors are stored externally as a cartridge. The cartridge's lower-layer K/V rows are appended to accumulated memory with rotary re-phasing; a forward pass with zero input embeddings injects the hidden state at the selected layer boundary, letting remaining frozen layers compute new memory representations in context of prior cartridges. Mistral path: FORGE → WRITE L0–L6 → INJECT H6 → REBUILD L7–L31 → ANSWER (Qwen: L0–L3/H3/L4–L27). Architectures differ (28 vs 32 layers, 3584 vs 4096 hidden, 4 vs 8 KV heads) so the procedure — not the cartridge — is what transfers. Two Zenodo research records (PDF, code, results) target researchers in activation steering/patching and memory-augmented transformers.
More from Research
- Stanford paper: generative representation learning cuts tokenizer training compute by 7-8x — HirokatuKataoka · 2026-10-16
- Cohere Labs to present confidence estimation for reasoning models at EMNLP 2026 — Cohere_Labs · 2026-10-16
- Vals RSI Index extrapolates Google models hitting 130% autonomous AI R&D by Oct 2027 — HaydnBelfield · 2026-10-16
- SpatialClaw Pulled Into Agentic Vision Data; Filtering and Evals Were the Real Lift — CMHungSteven · 2026-10-16
- Visual Lean: drag-and-drop GUI for the Natural Numbers Game runs Lean 4 in WASM — MilesCranmer · 2026-10-16
- Fine-Tuned 0.6B Router Beats Its 27B Sibling: Small Specialized Models Win — mshubham · 2026-10-16