Gradient Descent

Walking downhill on the loss surface: start from random weights, check which direction is downhill, take a small step, and repeat until the loss reaches a minimum. The Learning Rate sets the step size.

When to use

Use it to explain how a Neural Network's weights are adjusted in the training loop: forward pass, compute loss, adjust weights by gradient descent, next Epoch.

See Lec 8.

Appears in

No references yet.