Study
Deep Learning · Batch normalisation
Normalising each feature over the mini-batch, the learnable scale and shift, train vs eval mode, and its effect on a deep MLP.
- BatchNorm normalises each feature using the mini-batch mean and variance, then rescales with learnable γ and β.
- In eval mode BatchNorm uses running averages, so model.eval() is not optional.