Width vs Depth

Width is the number of neurons in one layer (more patterns at the same level of abstraction); depth is the number of hidden layers (higher-level abstractions built on lower ones). The deep in deep learning is depth.

When to use

Deeper networks learn more abstract patterns but need more data and training time. The lecture warns that bigger is not always better: for most business problems a 2-layer MLP with a few hundred neurons is plenty, and an oversized model wastes compute and invites Overfitting.

See DS Lec 8.

Appears in

No references yet.