Paper summary
Paper summary: Sharing is Caring: Efficient LM Post-Training with Collective RL Experience Sharing
SAPO is a fully decentralised, asynchronous RL post-training method in which many heterogeneous nodes train their own models but share rollouts, reporting up to 94% higher cumulative reward in controlled tests.
- Nodes don't share weights or gradients, only rollouts, so "aha moments" found by one node can spread to others.
- No assumptions about latency, identical models or identical hardware are needed.