FIM Mid-Training Boosts Coding Agent Performance

TIGER-Lab · hf · 2026-07-16

Proposes a mid-training approach for coding agents called function-aware fill-in-the-middle (FIM).

Core Idea

• Observes that a coding agent's "action-observation-reasoning" loop structurally resembles function calls in code.
• Thus, it uses program dependency graphs to select functions and applies dual-conditioned (complexity/inferability) self-supervised masking, enabling the model to learn function-level context completion during mid-training.

Training & Experiments

• Mid-trained Qwen2.5-Coder-Instruct 7B/14B and Qwen3-8B on 2.6B tokens of decontaminated corpus.
• Corpus sourced from 968 GitHub repositories.
• Subsequently integrated into existing agentic post-training pipelines for testing.

Results

• SWE-Bench-Verified improvements:
• 7B: +2.8
• 14B: +3.0
• Qwen3-8B: +3.2
• SWE-Bench-Lite improvements:
• 7B: +3.7
• 14B: +4.0
• Qwen3-8B: +5.4
• Gains hold across two post-training pipelines (R2E-Gym, SWE-Smith) and transfer to non-Qwen2.5 bases like Qwen3-8B + SWE-Lego.

Additional Findings

• This mid-training also mitigates capability degradation caused by agentic post-training.
• Effective beyond coding tasks, showing consistent gains on non-coding/tool-use benchmarks like LiveCodeBench, tau-bench, and BFCL.
• Despite training exclusively on Python, the inductive bias for function calls survives post-training.

Original post →

More from coding & agent

• A better path to agent autonomy is running waves, finding friction, and iterating — JnBrymn · 2026-07-22
• Coding agents are heading toward an AI-writes, AI-reviews, human-approves workflow — aftahi_ai · 2026-07-22
• oMLX 0.5.2 adds Mac menu-bar stats, low-bit decode kernels, and faster downloads — awnihannun · 2026-07-22
• GitHub review bot hits its PR limit and forces a 39-minute cooldown — DanielLockyer · 2026-07-22
• Max reasoning effort appears to be mobile-only in Codex Remote, not desktop — GabGarrett · 2026-07-22
• A Reddit demo argues online stores should expose carts and pricing through MCP — gelembjuk · 2026-07-22

coding & agent channel →