Function-Aware Fill-in-the-Middle mid-training improves coding agent foundation models

Researchers introduce function-aware fill-in-the-middle (FIM) as a mid-training objective for coding agent foundation models, leveraging the structural isomorphism between an agent's action-observation loop and function call sites. This self-supervised approach masks functions selected via program dependency graph analysis and applies to Qwen2.5-Coder-Instruct and Qwen3-8B models.

• Mid-training on a 2.6B-token corpus improves SWE-Bench-Verified scores by +2.8/+3.0 for 7B/14B models and +3.2 for Qwen3-8B.
• Gains on SWE-Bench-Lite reach +3.7, +4.0, and +5.4 respectively across the same model sizes.
• The method mitigates capability erosion in non-agent coding benchmarks like LiveCodeBench and tool-use tasks such as tau-bench and BFCL.

This technique demonstrates that function-call inductive biases survive agentic post-training, yielding consistent improvements across different pipelines and base models despite using only Python code for mid-training.