Lemmatization

Reducing a word to its true dictionary form (its lemma) using vocabulary and grammar, so the output is always a real word, for example better becomes good.

When to use

The last step of the text pipeline (lowercase, strip punctuation, tokenize, remove stopwords, lemmatize); PS 06 uses WordNetLemmatizer. It is slower but more accurate than Stemming, so use it for smaller datasets, and pick one method and stay consistent.

See Lec 7.

Appears in

No references yet.