Retro-DARC adds auditable depth-selective memory to frozen LLMs with a 262K adapter
broodsugar · x · 2026-07-28
Retro-DARC retrofits depth-selective memory into existing models without retraining
The post describes Retro-DARC, a Delta Attention Residual Compute adapter that aims to retrofit depth-selective memory into existing LLMs and world models.
Key claims from the thread:
- It is designed to work without retraining the base model.
- The adapter is inserted as an exact no-op at initialization: the author claims bitwise 0.0 deviation on three public checkpoints, including through MoE routing and early exit.
- Training is adapter-only, with exact rollback supported.
- The method also enables content-level memory audits, which weight-touching adapters supposedly cannot express by design.
- Early results show a roughly 262K-parameter adapter beating LoRA on a frozen public checkpoint, while being inserted with zero behavioral risk.
- The author further claims the memory bank can be zeroed to recover the frozen loss to machine precision, and that shuffling it destroys much of the gain.
More from Research
- As frontier models pass every human test, benchmarks may stop meaning much — julianvarascom · 2026-07-28
- Salesforce’s StateAct lifts Opus 4.8 on OSWorld 2.0 with state-first agents — Salesforce · 2026-07-28
- Netflix’s ID-V2V preserves identity while restyling videos from one source clip — netflix · 2026-07-28
- OpenAI economists get praise for work on how AI is changing jobs — soumitrashukla9 · 2026-07-28
- JEPA controller for PDEs cuts tracking error 53% on out-of-distribution targets — burny_tech · 2026-07-28
- Credit assignment, not better data or entropy, looks like the bottleneck for agentic RL — burny_tech · 2026-07-28