Why LLMs Remember but Fail to Apply After Fine-Tuning
HKUST · hf · 2026-07-13
This paper investigates a common issue in LLM fine-tuning: models rapidly memorize new knowledge but fail to apply it during downstream reasoning tasks.
The authors formalize this phenomenon as the Knowing–Using Gap, characterized by two aspects:
- The accuracy discrepancy between memorization and generalization.
- The time lag between a fact being memorized and becoming practically usable.
To explain the underlying mechanics, the paper introduces a self-patching intervention method. This tracks the spatial diffusion dynamics of knowledge within the model and identifies locations where migrating representations to the correct layers significantly resolves failed cases. The findings support a "knowledge circuit mismatch" hypothesis: although the knowledge exists internally, it is not routed to the layers engaged in computation. Finally, the authors propose a simple heuristic strategy that recovers about 58%–75% of the oracle margin in generalization failures.
Related event: Study Reveals Knowing-Using Gap in LLM Fine-Tuning(2 posts)→
More from Research
- EMNLP paper GENIE: a fine-grained metric locating where LLM stories are novel — gregd_nlp · 2026-10-27
- Hashed n-gram value embeddings push GPT to 3.2115 loss, beating 30M track record — RobertTLange · 2026-10-27
- MissionReport benchmark: frontier models score just ~60% on government status reporting — Exp_Mark · 2026-10-27
- TMLR paper: enforced thinking doesn't reliably make LLM agents disclose key info to users — SharonYixuanLi · 2026-10-27
- LLM-written Rust library bbtree verified: speed claims hold, memory savings reverse — lmoroney · 2026-10-27
- Diamandis: AI Cancer Navigator and $100 Genome Sequencing Close the Care Gap — PeterDiamandis · 2026-10-27