What is Elastic net regularization?
A regularization that adds both the L1 (lasso) and L2 (ridge) penalties to the loss, so one linear model gets both: L1 sets some weights to zero and L2 keeps that selection stable when features are correlated.
A regularization that adds both the L1 (lasso) and L2 (ridge) penalties to the loss, so one linear model gets both: L1 sets some weights to zero and L2 keeps that selection stable when features are correlated.
Card 1 of 263. Answer: A regularization that adds both the L1 (lasso) and L2 (ridge) penalties to the loss, so one linear model gets both: L1 sets some weights to zero and L2 keeps that selection stable when features are correlated.
About · Quizzes · History of AI · To suggest new questions or report an error send me a dm.
Elastic net adds a weighted mix of the L1 and L2 penalties to a model's loss: loss + λ₁ Σ |wᵢ| + λ₂ Σ wᵢ². So yes, a linear model can have both penalties at once. It keeps lasso's ability to set weights exactly to zero and adds ridge's smooth shrinkage, and lasso and ridge are its two extreme cases. Hui Zou and Trevor Hastie introduced it in 2005.
The L2 part fixes a lasso weakness. Suppose x₁ and x₂ are identical copies and the fit needs w₁ + w₂ = 2. Every non-negative split has L1 norm 2, so lasso may keep (2, 0), one copy only. The L2 penalty is 4 for (2, 0) but 2 for (1, 1), so elastic net shares the weight equally: the grouping effect.
The mechanism is clearest with squared loss ½ ‖y − Xw‖² and orthonormal features (XᵀX = I). Each weight then has a closed form: ŵ = sign(β̂) · max(|β̂| − λ₁, 0) / (1 + 2λ₂), where β̂ is the least-squares estimate. With λ₁ = λ₂ = 0.5, estimates of 4.0, 0.3 and −2.0 become 1.75, 0 and −0.75: lasso's thresholding, then ridge's proportional shrinkage.
Watch the naming: in scikit-learn, alpha is the overall strength and l1_ratio the L1 share (1 is lasso, 0 is ridge), while in R's glmnet, alpha is the L1 share and lambda the strength. Tune both by cross-validation, for example with ElasticNetCV, after scaling the features. It suits wide data such as gene expression, where features outnumber samples and come in correlated groups: lasso selects at most n features from n samples, while elastic net can keep more.