Paper summary: RRSI: Regularized Recursive Self-Improvement of Agent Harnesses

Problem

An LLM agent’s ability depends heavily on its harness: prompts, control flow, tools, memory and context management around a frozen model (see “Code as Agent Harness”, May 2026). Methods that automatically evolve harnesses create a kind of recursive self-improvement, but they can overfit, with big in-distribution gains that shrink or vanish out of distribution.

Method

RRSI applies regularisation ideas to harness evolution:

• the proposer has a temporally annealed budget on how many edits a candidate can bundle, and is pushed toward unexplored trajectories based on evolution history
• the selector has a critic that screens out benchmark-specific proposals, and a pruner that removes changes that are too small, too costly or no longer useful

Key results

From the abstract: across eight benchmarks (coding, agentic workspace, engineering design), up to +6.0 points on the split it evolves against and up to +4.3 on five out-of-distribution benchmarks.

Limitations

From the paper:

• only harness-level improvement with frozen models; no weight updates
• depends on a finite evolve set and several regularisation hyperparameters
• broader validation is needed across different agent architectures, tool ecosystems and longer self-improvement runs