Paper summary
Paper summary: Group Sequence Policy Optimization
Group Sequence Policy Optimization computes importance ratios and clipping at the level of whole sequences rather than tokens, and is reported to be more stable and efficient than GRPO, especially for MoE models.
- GRPO weights each token separately; GSPO uses the likelihood of the whole sequence for the ratio, clipping and reward.
- The authors credit GSPO for the improvements in the latest Qwen3 models.