Yes, because the nature of L1 regularization will lead to sparse coefficients of features. Feature selection can be done by keeping only features with non-zero coefficients.
Proposed by @Al_Grigor
Card 1 of 259. Answer: Yes, because the nature of L1 regularization will lead to sparse coefficients of features. Feature selection can be done by keeping only features with non-zero coefficients.
About · Quizzes · History of AI · To suggest new questions or report an error send me a dm.
Yes. L1 regularization, as in Lasso regression, adds the sum of the absolute values of the coefficients to the loss, and that penalty drives the coefficients of weak features to exactly zero. Keeping the features with non-zero coefficients is a built-in form of feature selection.
The reason is geometric. The L1 constraint region is a diamond with corners on the axes, and the loss contours usually first touch it at a corner, where some coefficients are exactly zero. L2 regularization uses a circle, so it shrinks coefficients toward zero but almost never to zero.
Two caveats come up in interviews. Scale the features first, since the penalty treats every coefficient the same. And when features are highly correlated, Lasso tends to keep one of them somewhat arbitrarily and drop the rest; Elastic Net, which mixes L1 and L2, spreads the weight across the group and gives more stable selections.
