ReLU
Also known as · Rectified Linear Unit
The Rectified Linear Unit activation, : it passes positive values through unchanged and turns negatives into zero.
When to use
Computed by hand in forward-pass questions, e.g. ReLU(1.1) is 1.1 and ReLU(-0.5) is 0. The lecture's standard recipe is ReLU in the Hidden Layers (fast and simple) and Sigmoid Function in the output layer when you need a probability.
See DS Lec 8.