None: regularization fights overfitting, and adding it to an underfitting model makes things worse. If the model is already regularized, weaken or remove it, and add capacity or better features.
Card 1 of 259. Answer: None: regularization fights overfitting, and adding it to an underfitting model makes things worse. If the model is already regularized, weaken or remove it, and add capacity or better features.
About · Quizzes · History of AI · To suggest new questions or report an error send me a dm.
None. Underfitting means the model is too constrained to capture the real pattern, a high-bias problem, and every regularizer, from L1 and L2 to dropout and early stopping, adds constraints that would push training error up further. If the model is already regularized, weaken or remove that regularization.
Ridge regression shows why. With orthonormal features and loss ‖y − Xw‖² + λ Σ wᵢ², each weight is its least-squares value divided by (1 + λ). If that value is 3 and λ = 9, the model estimates 0.3, a tenth of the effect, on training and validation data alike. Dropping λ to 0.1 gives about 2.73, and the underfitting disappears without changing the model's form.
Interviewers then ask what to do instead: add features or interactions, use a more flexible model, or train longer. The subtlety is that the cure can bring regularization back: a bigger network that fixes underfitting may then overfit and need weight decay or dropout. Judge the balance by training and validation error together.
