#activations
- 28 Jan 2025 · Study
Deep Learning · Activation functions
Sigmoid, tanh, ReLU, Leaky ReLU and GELU compared by shape and gradient, and why saturating activations make deep networks hard to train.
Sigmoid, tanh, ReLU, Leaky ReLU and GELU compared by shape and gradient, and why saturating activations make deep networks hard to train.