Bag of Words is a representation of text that describes the occurrence of words within a document.
Proposed by @Al_Grigor
Card 1 of 259. Answer: Bag of Words is a representation of text that describes the occurrence of words within a document.
About · Quizzes · History of AI · To suggest new questions or report an error send me a dm.
Bag of words represents a document by which words it contains and how often, ignoring grammar and word order. You build a vocabulary from the training texts, then turn each document into a vector with one entry per word, holding its count or a 1 or 0 for presence.
Example: 'the dog bit the man' and 'the man bit the dog' both become the: 2, dog: 1, bit: 1, man: 1, one vector for opposite meanings. Adding bigrams, pairs of adjacent words, separates them, since 'dog bit' appears only in the first. The cost is a bigger vocabulary; with 20,000 words a tweet fills perhaps 15 entries, so vectors are stored as sparse matrices.
With naive Bayes or logistic regression it is a strong baseline for spam and topic classification. Interviewers probe its limits: unseen words are dropped, synonyms such as 'good' and 'great' share nothing, and preprocessing matters. scikit-learn's CountVectorizer, for instance, lowercases text and by default drops one-character tokens such as 'a' and 'I'. TF-IDF reweights the counts.
