#initialisation
- 17 Feb 2025 · Study
Deep Learning · Weight initialisation
Why initial weight scale matters, Xavier/Glorot for tanh and He/Kaiming for ReLU, and watching activations explode or collapse through 50 layers.
Why initial weight scale matters, Xavier/Glorot for tanh and He/Kaiming for ReLU, and watching activations explode or collapse through 50 layers.