Paper summary: Qwen2 Technical Report
The Qwen2 report covers open dense models from 0.5B to 72B plus an MoE, with Qwen2-72B reaching 84.2 on MMLU as a base model and support for about 30 languages.
Problem
Open-weight models had been catching up with proprietary ones. This report documents Alibaba’s next generation, Qwen2, across a wide range of sizes.
Method
The Qwen2 series includes foundation and instruction-tuned models from 0.5B to 72B parameters, with dense models and a mixture-of-experts model. The report covers architecture, pre-training and post-training.
Key results
As reported in the abstract:
- Qwen2-72B (base): 84.2 MMLU, 37.9 GPQA, 64.6 HumanEval, 89.5 GSM8K, 82.4 BBH
- Qwen2-72B-Instruct: 9.1 MT-Bench, 48.1 Arena-Hard, 35.7 LiveCodeBench
- surpasses most prior open-weight models, including Qwen1.5, and is competitive with proprietary models
- multilingual ability in about 30 languages
Limitations
There’s no dedicated limitations section in the extracted text. Benchmarks are reported by the model’s own developers, which is standard for technical reports but worth keeping in mind.
Sources
- Qwen2 Technical Report (arXiv:2407.10671) — arXiv
- Hugging Face Daily Papers, 2024-07-16 — Hugging Face
- Qwen2 Technical Report (arXiv HTML full text) — arXiv
- Code repository (linked in the paper) — GitHub
This is a summary written from the linked public sources. Figures and numbers are as reported there.