Paper summary
Paper summary: QeRL: Beyond Efficiency -- Quantization-enhanced Reinforcement Learning for LLMs
QeRL combines NVFP4 quantisation with LoRA for RL training of LLMs; beyond saving memory and speeding up rollouts, quantisation noise raises policy entropy and helps exploration. It enables RL on a 32B model with one H100.
- The noise that quantisation adds turned out to help RL, by raising entropy and encouraging exploration.
- The paper reports the first RL training of a 32B LLM on a single H100 80GB GPU.