Deep Learning
One hot encoding is an excellent solution to transform categorical features with high cardinality.
False
Proposed by @svpino
Card 1 of 276. Answer: False
False
Proposed by @svpino
Card 1 of 276. Answer: False
About · Quizzes · History of AI · To suggest new questions or report an error send me a dm.
False. One-hot encoding turns each value of a categorical feature into its own binary column. It suits low-cardinality features, but a high-cardinality feature, such as product or user IDs, has thousands of distinct values, so it becomes thousands of mostly-zero columns.
Take a product_id column with 50,000 values, many seen only a few times. Feeding 50,000 one-hot columns into a 128-unit dense layer means 6.4 million weights, and those for a rare product are learned from a handful of rows, so they mostly fit noise. A product first seen at prediction time becomes an all-zero row. Trees suffer too, since each split isolates one product.
Better options: an embedding layer, a short learned dense vector per category (16 numbers per product is 800,000 weights), out-of-fold target encoding, hashing, or one catch-all category for rare values. An embedding is mathematically a one-hot vector times a weight matrix, computed as a lookup. Very wide one-hot features do work in sparse linear models trained on huge data, as in ad click prediction.