Outliers

Values far from the bulk of the data, flagged by the IQR rule: x<Q1−1.5 IQRx < Q_1 - 1.5\,\text{IQR} or x>Q3+1.5 IQRx > Q_3 + 1.5\,\text{IQR}, where IQR=Q3−Q1\text{IQR} = Q_3 - Q_1. They can distort the mean and other summaries and models.

When to use

Spot them with a boxplot, then investigate before removing: an 18,000 dollar order in a 100 to 3,000 dollar dataset may be an error or your biggest customer. The median and MAE are less sensitive to them; K-Means and PCA are sensitive.

See DS Lec 2.

Appears in