IAR Framework: 3-Stage Post-Training for Retrieval-Free Knowledge Internalization
_reachsumit · x · 2026-08-21
The paper presents IAR (Inject, Align, Recover), a three-stage post-training framework designed to internalize document collections into parametric knowledge for retrieval-free QA. The method consists of:
- Inject: Converting source documents into continuation, rewrite, and instruction-conditioned reconstruction objectives.
- Align: Adapting the injected model with answer-only QA supervision.
- Recover: Merging the domain-adapted model with the base instruction model to restore general capabilities.
Experiments on Llama, Phi, Qwen, and SmolLM show that IAR improves domain QA accuracy by an average of 3.6 percentage points and general performance (IFEval, MMLU, MSBench) by 12.1 percentage points, outperforming Vanilla SFT.
Related event: BAAI Unveils IAR Framework for Retrieval-Free Knowledge Injection(2 posts)→
More from Research
- Berkeley paper uses the Midas myth to rethink robotics beyond human imitation — begusgasper · 2026-10-16
- 31 Tech Curves Visualized: Kurzweil's $1,000-Brain-Compute Call Missed by 400x — cpaik · 2026-10-16
- DeepMind Paper: First-Price Auctions in Agentic Markets Reach 1-1/e^2 Efficiency — dair_ai · 2026-10-16
- View: induction heads fully explain why transformers won and why RL is the right algorithm — akbirthko · 2026-10-16
- SpliceAI2 benchmarks show major gains across nine splicing-variant evaluations — anshulkundaje · 2026-10-16
- GRPO post-training collapses model distributions, clashing with long-horizon agents — zsakib_ · 2026-10-16