L1 penalizes the absolute magnitude of the coefficients while L2 penalizes the square of the coefficients.
Card 1 of 259. Answer: L1 penalizes the absolute magnitude of the coefficients while L2 penalizes the square of the coefficients.
About · Quizzes · History of AI · To suggest new questions or report an error send me a dm.
L1 regularization (lasso) adds λ Σ |wᵢ|, the sum of the weights' absolute values, to the loss; L2 (ridge) adds λ Σ wᵢ², the sum of their squares. As a result, L1 sets some weights exactly to zero, selecting features, while L2 shrinks every weight but keeps them all.
The penalties charge weights differently. For weights (4, 0.2), L1 charges 4.2 and L2 charges 16 + 0.04 = 16.04. Under L2 the small weight costs almost nothing, so nothing pushes it out. Under L1 it costs the same per unit as the large one, so if its feature barely reduces the error, zeroing it is cheaper. In Bayesian terms, L1 is the MAP estimate under a Laplace prior, L2 under a Gaussian.
Ridge has a closed-form solution and spreads weight across correlated features; lasso generally needs iterative solvers and tends to keep one feature of a correlated group. Don't confuse them with L1 and L2 losses, mean absolute and mean squared error, which measure prediction error, not weight size.
