Paper summary: DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models

Problem

Competition-level maths is hard for language models, and the strongest results came from closed models like GPT-4 and Gemini Ultra.

Method

Two ingredients:

1. Data: an engineered pipeline selects 120B maths-related tokens from Common Crawl. The model, starting from DeepSeek-Coder-Base-v1.5 7B, continues pre-training on these together with natural-language and code data (the paper says 500B tokens in total).
2. GRPO (Group Relative Policy Optimization): a PPO variant for the RL stage. Instead of training a separate value model as a baseline, it samples a group of answers per question and uses their relative rewards. This improves reasoning while using less memory than PPO.

Key results

From the abstract:

• 51.7% on the competition-level MATH benchmark without external tools or voting, approaching Gemini-Ultra and GPT-4
• 60.9% with self-consistency over 64 samples

The ablations report that web pages are a strong source of maths data, while arXiv papers were less useful than the authors expected.

Limitations

The paper’s closing section summarises results rather than listing limitations. The headline is on maths benchmarks specifically. GRPO is presented as effective even after the instruct model already scores highly.