Paper summary: Large Language Models as Optimizers

Problem

Many optimisation problems have no gradients. Prompt engineering is one of them: the “parameters” are words, and the objective is task accuracy.

Method

Optimization by PROmpting (OPRO) writes the optimisation problem in plain language. Each step:

1. A meta-prompt describes the task and lists previously generated solutions with their scores, sorted.
2. The LLM proposes new candidate solutions.
3. The candidates are evaluated and added to the meta-prompt for the next step.

The authors first try it on linear regression and small travelling-salesman problems, then focus on prompt optimisation: searching for an instruction (e.g. a phrase prepended to a maths question) that maximises accuracy.

Key results

Across several LLMs, the paper reports that OPRO-optimised prompts outperform human-designed prompts by up to 8% on GSM8K and up to 50% on Big-Bench Hard tasks. On small travelling-salesman instances, the conclusion notes OPRO performs on par with some hand-crafted heuristics.

Limitations

The paper is explicit that OPRO isn’t meant to beat gradient-based optimisers or dedicated solvers. The length of the LLM’s context window makes large problems (high-dimensional regression, travelling salesman with many nodes) hard to fit in the prompt. On larger TSP instances, a classic farthest-insertion heuristic overtakes the LLMs. The paper also includes a section analysing overfitting in prompt optimisation.