Paper summary: Group Sequence Policy Optimization
Group Sequence Policy Optimization computes importance ratios and clipping at the level of whole sequences rather than tokens, and is reported to be more stable and efficient than GRPO, especially for MoE models.
Problem
RL algorithms for LLMs, like GRPO (introduced in the DeepSeekMath paper, summarised here in Feb 2024), compute importance ratios per token. The authors argue this can make training unstable, especially for Mixture-of-Experts models.
Method
GSPO defines the importance ratio from the sequence likelihood, and does clipping, rewarding and optimisation at the sequence level. The unit of reward (a whole response) now matches the unit of optimisation.
Key results
From the abstract: GSPO has better training efficiency and performance than GRPO, notably stabilises MoE RL training, and may simplify RL infrastructure. The authors credit it for the improvements in the latest Qwen3 models.
Limitations
There’s no dedicated limitations section in the extracted text. The evidence comes from the authors’ own Qwen training runs.
Sources
- Group Sequence Policy Optimization (arXiv:2507.18071) — arXiv
- Hugging Face Daily Papers, 2025-07-25 — Hugging Face
- Group Sequence Policy Optimization (arXiv HTML full text) — arXiv
This is a summary written from the linked public sources. Figures and numbers are as reported there.