Text Preprocessing
Cleaning raw text into tidy tokens before modelling, in the order: lowercase, remove punctuation, Tokenization, remove Stopwords, then Stemming or Lemmatization.
When to use
Asked as a step-by-step exercise on a sample review, e.g. the Bikoret exercise review (9 words reduced to 5 content tokens). It feeds Bag of Words and the Document-Term Matrix. Each step has a pitfall: lowercasing loses Apple the company versus apple the fruit, and stopword removal can destroy negation. Also in PS 06.
See DS Lec 7.