Machine Learning
Increasing model complexity tends to decrease bias and increase variance, which leads to overfitting. True or False?
True.
Card 1 of 259. Answer: True.
True.
Card 1 of 259. Answer: True.
About · Quizzes · History of AI · To suggest new questions or report an error send me a dm.
True. Making a model more complex, with more parameters, deeper trees or higher-degree features, tends to lower bias and raise variance. Bias is error from overly rigid assumptions; variance is how much the fitted model changes with the training sample. Once variance dominates, the model overfits.
Expected test error = bias² + variance + irreducible noise. In k-nearest neighbors, k = 50 averages many points, giving smooth, stable, biased predictions; k = 1 copies the nearest training point, so bias is low but predictions jump with each new sample. As complexity grows, test error usually falls, then climbs once variance rises faster than bias drops: a U-shape.
Interviewers often ask how to find the bottom of the U: cross-validate over a complexity knob such as tree depth, k or regularization strength. One known exception is double descent: heavily overparameterized models such as large neural networks can see test error fall again after they fit the training data perfectly, so parameter count alone is a poor measure of complexity.