Paper summary: Qwen2.5-Coder Technical Report

Problem

Code models need strong programming ability, but specialising often erodes general and maths skills. This report upgrades the earlier CodeQwen1.5.

Method

Qwen2.5-Coder comes in six sizes (0.5B, 1.5B, 3B, 7B, 14B, 32B), built on the Qwen2.5 architecture and continued-pretrained on over 5.5 trillion tokens. The emphasis is on the data pipeline:

• careful data cleaning
• scalable synthetic data generation
• balanced mixing of code, maths and general text, to keep non-code skills

Key results

From the abstract: state-of-the-art performance across more than 10 benchmarks covering code generation, completion, reasoning and repair, consistently outperforming larger models. The models are released under a permissive licence.

Limitations

There’s no dedicated limitations section in the extracted text. As with other technical reports, the benchmark results are reported by the model’s developers.