Study
Machine Learning · Gradient descent
Fitting linear regression by gradient descent from scratch: the gradient of MSE, the learning rate, and what happens when it is too small or too big.
- Gradient descent repeatedly steps opposite the gradient, the direction of steepest increase in loss.
- Too small a learning rate crawls; too large a rate diverges.