IAR Framework: 3-Stage Post-Training for Retrieval-Free Knowledge Internalization
_reachsumit · x · 2026-08-21
The paper presents IAR (Inject, Align, Recover), a three-stage post-training framework designed to internalize document collections into parametric knowledge for retrieval-free QA. The method consists of:
- Inject: Converting source documents into continuation, rewrite, and instruction-conditioned reconstruction objectives.
- Align: Adapting the injected model with answer-only QA supervision.
- Recover: Merging the domain-adapted model with the base instruction model to restore general capabilities.
Experiments on Llama, Phi, Qwen, and SmolLM show that IAR improves domain QA accuracy by an average of 3.6 percentage points and general performance (IFEval, MMLU, MSBench) by 12.1 percentage points, outperforming Vanilla SFT.
Related event: BAAI Unveils IAR Framework for Retrieval-Free Knowledge Injection(2 posts)→
More from Research
- Proposal: Release Failed RL Checkpoints as Better 'Model Organisms' for Safety Research — CFGeek · 2026-08-24
- TSUI: Native UI Framework Compiling TS/XML to GPU — johnlindquist · 2026-08-24
- Anthropic Study: Fine-Tuned Lie Detectors Fail to Generalize OOD — PandaAshwinee · 2026-08-24
- Shengshu Tech Unveils 5-Stage Roadmap for General World Models — 生数科技 · 2026-08-24
- AGI May Arrive First in Hard Tech Due to Objective Feedback Loops — imjustnewatai · 2026-08-24
- Trained a 1.57B-parameter Dreamer 4 World Model from scratch for under $150 — OtherRaisin3426 · 2026-08-24