Paper summary
Paper summary: Qwen2.5 Technical Report
Qwen2.5 scales pre-training data from 7 to 18 trillion tokens and uses over a million SFT samples plus multistage RL, across a wide range of open-weight sizes.
- Pre-training data grew from 7T to 18T tokens.
- Post-training combines more than 1 million SFT samples with multistage reinforcement learning.