Study
Deep Learning · Dropout
Dropout randomly zeroes activations during training as a regulariser; inverted scaling, train vs eval behaviour, and its effect on an overfitting MLP.
- Dropout zeroes each activation with probability p during training and scales the rest by 1/(1−p).
- At evaluation time dropout is switched off, which is another reason model.eval() matters.