Learning a fact isn't learning how to retrieve it: request-form training shapes LLM recall
_reachsumit · x · 2026-10-05
An arXiv paper (2610.03251) separates learning a fact from learning how to retrieve it. A model trained on "The capital of X is Y" may answer after "The capital of X is" but fail after "The capital of X:" — different request forms elicit the same fact differently. Two-stage experiments show a model pretrained on five request forms later retrieves identical facts well across forms, while a statement-only model doesn't — meaning retrieval skills are learned through request forms before facts themselves. The authors trace this to the hidden "context state" right before the answer: intervening on it at retrieval or training time can enable or block recall of learned facts, with effects transferring across facts and relations like capitals and currencies.
More from Research
- OpenTac auto-routes tactile sensor PCBs from a 3D model in 3 seconds, covering 93% of surface — anand_bhattad · 2026-10-23
- arXiv paper gives StopGrads theoretical grounding: convergence without an underlying loss — srush_nlp · 2026-10-23
- PEARL demo: an LLM agent that checks experiment results, revises plans, and re-tests — KevinKaichuang · 2026-10-23
- Berkeley's StyleStream 2.0 cuts streaming voice conversion latency to 520ms, 2x faster — Berkeley · 2026-10-23
- Math PhD student quits academia: AI is making math unrecognizable — pickover · 2026-10-23
- ICLR Submission Found Containing LLM's Own Meta-Comment Admitting Results Weren't Significant — _onionesque · 2026-10-23