Paper summary
Paper summary: SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training
On a card-arithmetic game and a real-world navigation task, models post-trained with RL generalise to unseen rule and visual variants, while SFT tends to memorise the training data.
- With an outcome-based reward, RL generalised to new rules and new visuals; SFT mostly didn't.
- SFT was still useful: it helped stabilise the output format before RL.