Paper summary
Paper summary: GDPO: Group reward-Decoupled Normalization Policy Optimization for Multi-reward RL Optimization
With several rewards (correctness, format, length…), GRPO's group normalisation can collapse different reward combinations into identical advantages; GDPO normalises each reward separately and trains more stably.
- Normalising the summed reward can give the same advantage to quite different outcomes, which blurs the training signal.
- Decoupling the normalisation of each reward keeps their relative differences.