Paper summary: START: Self-taught Reasoner with Tools

Problem

Large reasoning models (like o1 and DeepSeek-R1) think in long chains of thought, but only internally. That leads to hallucinated calculations and inefficiency.

Method

START adds a Python interpreter to long chain-of-thought reasoning, learned through self-training:

1. Hint-infer: insert hand-designed hints into the model’s reasoning at inference time, such as “Wait, maybe using Python here is a good idea.” This alone triggers tool use, with no demonstrations, and also works as a simple test-time scaling method.
2. Hint-RFT: generate tool-using trajectories with Hint-infer, then score, filter and modify them, and fine-tune the model on the result (rejection-sampling fine-tuning).

They apply this to QwQ-32B.

Key results

From the abstract: 63.6% on GPQA, 95.0% on AMC23, 66.7% on AIME24, 47.1% on AIME25 and 47.3% on LiveCodeBench. That’s significantly above base QwQ-32B and comparable to R1-Distill-Qwen-32B and o1-Preview.

Limitations

From the paper:

• the Python interpreter is the only tool; search engines or other tools are left for future work
• the manually designed hints may disrupt the model’s natural flow of thought