Writing Knowledge into LLMs Isn't Just About Capacity
menhguin · x · 2026-07-16
This post dives into a technical debate on how to write new knowledge into LLM weights without destroying existing behaviors.
The core argument is that approximate orthogonality in feature space, update directions in parameter space, and behavioral changes in function space shouldn't be conflated. Even if new and old features are nearly orthogonal in representation, the parameter updates required for new capabilities might not fully decouple from existing computations. The author formalizes the constraint linearly: the update \\(\Delta\theta\\) must satisfy the new task objective \\(J{new} \times \Delta\theta = r\\) while keeping old behaviors unchanged, i.e., \\(J{old} \times \Delta\theta = 0\\).
The counterargument is that massive models have countless nearly orthogonal directions, providing enough capacity to write new knowledge "elsewhere," making this an engineering problem rather than a research one. The original poster leans toward refuting this oversimplification: even with a massive nullspace, the real challenge lies in locating that space, routing new representations into it, and connecting them to the correct downstream computations, rather than just checking if there is "enough capacity."
Related event: Technical Debate on Writing New Facts into LLM Weights(3 posts)→
More from AGI Musings
- A multipolar AI race will not automatically make AI go well, repost argues — JeffLadish · 2026-07-22
- Humanoid robot sorting packages in a warehouse sparks debate over job loss — MonaJalal_ · 2026-07-22
- You can outsource thinking, but not understanding, in the age of agents — Yuchenj_UW · 2026-07-22
- India’s multilingual LLM edge, once obvious, is gone, the post argues — kmeanskaran · 2026-07-22
- AI media may be cleaned up with provenance tracking, notes, and prediction markets — NathanpmYoung · 2026-07-22
- Ryan Greenblatt says economists underestimate AI’s growth impact even in a 100 million worker scenario — RyanGreenblatt · 2026-07-22