A regularization technique that linearly combines the L1 (lasso) and L2 (ridge) penalties.
Card 1 of 259. Answer: A regularization technique that linearly combines the L1 (lasso) and L2 (ridge) penalties.
About · Quizzes · History of AI · To suggest new questions or report an error send me a dm.
Elastic net regularization adds a weighted mix of the L1 and L2 penalties to a model's loss: loss + λ₁ Σ |wᵢ| + λ₂ Σ wᵢ². It keeps lasso's ability to set weights exactly to zero and adds ridge's smooth shrinkage, which keeps selection stable when predictors are correlated. Lasso and ridge are its two extreme cases.
The mechanism is clearest with squared loss ½ ‖y − Xw‖² and orthonormal features (XᵀX = I). Each weight then has a closed form: ŵ = sign(β̂) · max(|β̂| − λ₁, 0) / (1 + 2λ₂), where β̂ is the least-squares estimate. With λ₁ = λ₂ = 0.5, estimates of 4.0, 0.3 and −2.0 become 1.75, 0 and −0.75: lasso's thresholding, then ridge's proportional shrinkage.
Watch the naming: in scikit-learn, alpha is the overall strength and l1_ratio the L1 share, while in R's glmnet, alpha is the L1 share and lambda the strength. It suits wide data such as gene expression, where features outnumber samples and come in correlated groups.
