Lasso, short for least absolute shrinkage and selection operator. It adds λ times the L1 norm of the weights, λ Σ |wᵢ|, to the loss, which drives some weights exactly to zero.
Card 1 of 259. Answer: Lasso, short for least absolute shrinkage and selection operator. It adds λ times the L1 norm of the weights, λ Σ |wᵢ|, to the loss, which drives some weights exactly to zero.
About · Quizzes · History of AI · To suggest new questions or report an error send me a dm.
L1 regularization is also called lasso (least absolute shrinkage and selection operator), the name of L1-penalized linear regression. It adds λ Σ |wᵢ|, λ times the L1 norm of the weights, to the loss. The L1 norm is the quantity penalized, not another name for the method. L2 is ridge.
Robert Tibshirani introduced it in 1996. It sets some weights exactly to zero, selecting features. With orthonormal features and loss ½ ‖y − Xw‖² + λ Σ |wᵢ|, each weight is its least-squares value shrunk by λ and clipped at zero: ŵ = sign(β̂) · max(|β̂| − λ, 0). With λ = 0.5, coefficients 3.0, 0.4 and −1.2 become 2.5, 0 and −0.7.
The usual follow-up: why does L1 give zeros and L2 not? The L1 penalty pushes with constant force λ all the way to zero, while the L2 push, 2λw, fades as w shrinks, so weights get small but stay nonzero. Among correlated features lasso keeps one arbitrarily, which is why elastic net mixes L1 and L2.
