Gradient Descent
Walking downhill on the loss surface: start from random weights, check which direction is downhill, take a small step, and repeat until the loss reaches a minimum. The Learning Rate sets the step size.
When to use
Use it to explain how a Neural Network's weights are adjusted in the training loop: forward pass, compute loss, adjust weights by gradient descent, next Epoch.
See Lec 8.