Machine Learning
Can we have both L1 and L2 regularization components in a linear model?
Yes, it is called Elastic net regularization.
Card 1 of 259. Answer: Yes, it is called Elastic net regularization.
Yes, it is called Elastic net regularization.
Card 1 of 259. Answer: Yes, it is called Elastic net regularization.
About · Quizzes · History of AI · To suggest new questions or report an error send me a dm.
Yes. A linear model with both penalties is called elastic net: the loss is the usual error plus λ₁ Σ |wᵢ| + λ₂ Σ wᵢ². The L1 part sets some weights exactly to zero; the L2 part shrinks weights smoothly and stabilizes the solution. Hui Zou and Trevor Hastie introduced it in 2005.
The L2 part fixes a lasso weakness. Suppose x₁ and x₂ are identical copies and the fit needs w₁ + w₂ = 2. Every non-negative split has L1 norm 2, so lasso may keep (2, 0), one copy only. The L2 penalty is 4 for (2, 0) but 2 for (1, 1), so elastic net shares the weight equally: the grouping effect.
In scikit-learn, ElasticNet takes alpha, the overall strength, and l1_ratio, the L1 share: 1 is lasso, 0 is ridge. Tune both by cross-validation, for example with ElasticNetCV, after scaling the features. It also helps when features outnumber rows: lasso selects at most n features from n samples, while elastic net can keep more.