Paper summary: Scaling Automatic Research Agents via World Models

Problem

AutoResearch agents implement solutions and learn from execution results, and RL post-training is central to their gains. But the two parts of each trajectory scale differently (Figure 1). Generation shares compute through batching, while execution occupies its own sandbox and real machine time. As trajectories grow, execution dominates training cost.

Method

World Model RL (WMRL) replaces environment execution with a world model that predicts outcomes, a close relative of the language world models summarised here in June 2026. Because those predicted rewards are biased and noisy, WMRL adds:

• Online Debiasing, to offset bias
• Inverse-Variance Denoising, to suppress noise

The authors prove that both mitigations strictly improve the convergence guarantee.

Key results

From the abstract: 3–4× faster training across tasks and agent scales, while beating standard RL baselines. The post-trained 4B and 9B agents outperform much larger models, per the authors.

Limitations

There’s no dedicated limitations section in the extracted text. The approach depends on how well the world model approximates real execution, which is the problem the two mitigations target.