DEFT extends its Core–Slack split across layers to keep transfer learning stable
TheGradient · x · 2026-07-24
Layer-to-layer routing
This installment explains how DEFT propagates the Core/Slack split across layers. Core neurons connect to Core neurons, Slack to Slack, while cross-links are severed at time step zero for the forbidden directions.
The intended effect
By preserving only the allowed routes, the method tries to keep foundational features stable while letting new learning accumulate in the plastic branch. In the thread’s framing, this is what makes the system usable for continual learning and transfer learning rather than just compression.
Related event: DEFT Introduces Core-Slack Architecture to Prevent Catastrophic Forgetting(3 posts)→
More from Research
- Lab monkeys at $30-80K each: a 4-year breeding cycle is the real preclinical drug bottleneck — zebird0 · 2026-10-27
- OpenAI Foundation funds UNC tumor antigen project, hiring senior experimental scientist — iskander · 2026-10-27
- Steering attention heads toward honesty: up moral patienthood, down psychopathy — edelwax · 2026-10-27
- Are transcoders overkill? New paper argues plain neurons suffice for circuit analysis — burny_tech · 2026-10-27
- EMNLP paper GENIE: a fine-grained metric locating where LLM stories are novel — gregd_nlp · 2026-10-27
- Hashed n-gram value embeddings push GPT to 3.2115 loss, beating 30M track record — RobertTLange · 2026-10-27