Paper summary
Paper summary: Function-Aware Fill-in-the-Middle as Mid-Training for Coding Agent Foundation Models
The authors note that an agent's act–observe–continue loop has the same shape as a function call site, and use function-level fill-in-the-middle mid-training on Python repos to improve coding agents on SWE-Bench.
- Calling a tool and using its result looks like calling a function and using its return value, and ordinary code has billions of examples of that.
- Masking whole functions and asking the model to fill them in teaches it to use information that arrives from elsewhere.