Learning a Fact Is Not Learning How to Retrieve It
Chaemin Jang, Jihee Kim, Dongman Lee
cs.AI
2026-10-02
Two-stage fine-tuning separates learning a fact from retrieving it: statement accuracy stays near 100%, while list-form retrieval of new facts goes from 0% to 97.7%.
Fine-tune on 'The capital of X is Y' and the model can emit Y when the prompt keeps that wording. End the prompt with a colon and Y disappears. Same fact, different shell.
Allen-Zhu and Li (2024) and Jiang et al. (2024) had already seen related gaps. Request-form data in pretraining, or instruction tuning before continued pretraining, changes whether later text can be pulled back out. Ordinary fine-tuning learns the fact and its wording together, so a miss has two readings: the fact is weak, or the model never learned the second way of asking. The KAIST setup splits those two.
A request form is one shell around the same fact. The main runs use Pythia-410M and fictional country-capital pairs: 1,000 facts in stage one, 1,000 new facts in stage two, disjoint as strings and as capital first tokens. Evaluation is a fixed 300 facts, averaged over eight paired seeds.
In stage one the five-form model sees each old fact as a statement, a key-value line, a question, a dialogue turn, and an imperative. The statement-only model sees the same facts as statements only, with the token count matched. In stage two both models train on byte-identical new facts, every line written as 'The capital of X is Y'. Of the eight test forms, only that declarative line ever appears in training. Scoring is pool top-1, whether the right capital ranks first among 2,000 candidates, and exact generation, whether greedy decoding starts with the capital.
The context state is the hidden state just before the answer. Retrieval tracks the words next to that position. On the statement-only model, changing a list prompt from a colon ending to an 'is' ending moves exact generation from 0.2% to 81.4%. A 30-token unrelated preamble does not change which forms work.
Five relations share one training run, 200 target facts each. Both models sit near the ceiling on the declarative training form. The list form splits them. Birth years are higher on the statement-only side, because a year is already a likely continuation after a colon.
| Relation | Declarative, statement-only / five-form | List, statement-only / five-form |
| Capital | 98.3% / 99.9% | 0.2% / 97.0% |
| Currency | 99.5% / 99.9% | 0.4% / 98.9% |
| Population | 99.1% / 98.6% | 3.8% / 99.1% |
| Founder | 99.2% / 99.2% | 0.1% / 98.9% |
| Birth year | 98.7% / 98.6% | 34.4% / 98.2% |
Capitals alone, eight forms: declarative 99.5% vs 99.7%, list 0% vs 97.7%, colon-ended declarative 2.4% vs 99.3%. Possessive and short key-value forms share no token bigram with any stage-one form and reach only 16.5% and 22.7%. A relative-clause paraphrase falls from 89.5% to 80.1%. The facts themselves are not learned better. First-token negative log-likelihood differs by -0.043 nats, the margin over the strongest wrong answer by +0.24, and both intervals include zero. Extending stage two to 4,800 updates, or cutting the learning rate by 5x until declarative accuracy matches, leaves list exact-generation at 0.
Stage one still transfers when its answers are wrong or its relation changes. Mean accuracy on the two held-out forms is 1.2% after statements only, 68.1% after each country is paired with another country's capital, and 94.4% after currency facts. Writing 5% of the new facts in the five forms, and the rest as statements, lifts list accuracy to 38.1%. The same fraction written as declarative paraphrases yields 0.3%.
Swapping the context state toggles retrieval. Replacing the list-form state with the declarative state raises exact generation from 0 to 96.3%, close to rewriting the prompt so it ends in 'is' (94.8%). The reverse swap drops the declarative form from 99.7% to 2.3%. A norm-matched random vector does nothing, and patching the country-name token does nothing. At layer 4, a state from the same fact restores 96.3%, from another fact 95.1%, from currency 94.6%, from population 92.8%. An entity-free state still scores 73.4% at layer 4 and about 2% at layer 6. States that carry an entity stay above 70%.
A single direction does most of this work. The difference between 'The value of {c} is' and 'The value of {c}:' carries no capital relation. Added to a failing list prompt, it moves accuracy from 0 to 92.1%. Projecting it out of the full restoring vector drops accuracy to 1.0%. A fitted linear map reaches only 94.9%. The same direction lifts the colon-ended form from 2.4% to 98.3%. Across domains versus within a domain, capitals score 91.0% against 92.1%, and birth years 96.9% against 97.1%.
Shifting the state only while the fact is learned, then testing with no intervention, changes later retrieval. Training lines stay declarative. Colon-ended accuracy goes from 2.4% to 94.3% on Pythia-410M, where a random shift reaches 10.0%, from 26.6% to 90.4% on Pythia-1.4B, from 48.2% to 99.3% on Qwen2.5-1.5B, and from 24.6% to 98.3% on birth years. Writing 5% of the target facts in the new forms reaches 72% on that colon-ended form. The shift reaches 94% without rewriting any target fact. Qwen already scores 60.3% on the list form before any intervention, and removing the direction from its restoring vector still leaves 94.4%. On Pythia, removing it removes the restoration.
A test that only uses the training sentence scores both models near 100% and writes a retrieval miss down as a failure to learn. A miss on one form also does not mean the fact is absent from the weights. Changing the state just before the answer brings it back.
For knowledge injection, request forms can be learned first and reused on later facts, with wrong answers or with a different relation such as currency. Shifting the context state during fine-tuning avoids rewriting every new fact into several shells.
The edit sits on the residual stream at a chosen layer, and it needs a target state. That is a mechanism result, not a fine-tuning recipe ready to paste into a training run.
The paper states three limits. The facts are synthetic and the two stages are separated on purpose, so learning from natural text may use other mechanisms. The context state does not explain every failure to retrieve a fact. Strength depends on what the base model already does with a request form, so scale, pretraining mix, and instruction tuning can change it.
A few gaps sit on top of that. In the main capital pool every answer has a unique first token among 2,000 candidates, which real knowledge does not. Five-form experience does not move shells that share no token bigram with the old forms: possessive and short key-value stop at 16.5% and 22.7%. The high birth-year baseline shows surface co-occurrence leaking in. On Qwen a random shift alone moves colon-ended accuracy from 48.2% to 72.2%. The models stop at base checkpoints from 410M to 1.5B. No instruction-tuned chat model, and no continued pretraining on real documents.