Machine Learning interview questions

Models, metrics and the tradeoffs that decide whether a model survives contact with real data.

82 questions in this category.

The cards below cluster into the themes machine learning interviews keep circling back to: the bias-variance tradeoff and the regularization that moves it, picking metrics when the classes are imbalanced and reading them right, from sensitivity and specificity to calibration, log loss and ranking scores like NDCG, cross-validation and the leakage that quietly inflates it, ensembles from bagging to boosting, and the encoding choices that decide what a model ever gets to see.

Name three supervised learning algorithms.

Any three of: linear regression, logistic regression, decision trees, random forests, gradient boosting, support vector machines, k-nearest neighbors, naive Bayes and neural networks.

Name three unsupervised learning algorithms.

k-means clustering, principal component analysis (PCA) and DBSCAN; others include hierarchical clustering, Gaussian mixture models, isolation forest and autoencoders. Clustering and anomaly detection are tasks rather than algorithms.

What is the type of machine learning algorithms used to draw inferences from labeled data?

Supervised Learning.

What is the type of machine learning algorithms used to draw inferences from data without labels?

Unsupervised Learning.

Which graph plots the true positive rate against the false positive rate at various classification thresholds?

ROC curve.

Name three types of kernels in SVM.

Linear, polynomial and RBF (radial basis function), with sigmoid a fourth. A kernel computes inner products in a higher-dimensional space without ever building that space, so linear is the default for wide sparse text and RBF for low-dimensional, curved boundaries.

What is pruning in a decision tree?

Removing sub-nodes that do not earn their keep, so a fully grown tree stops fitting noise. Pre-pruning caps growth up front with max depth or minimum samples per leaf; post-pruning grows the tree, then collapses branches whose removal does not hurt held-out error.

What is ensemble learning?

Combining individual models together with the purpose of improving the predictive power of the model.

Would you use k-fold cross validation on time series data? Explain.

Not standard shuffled k-fold: time series rows are ordered and autocorrelated, so random folds train on the future to predict the past. Use time-ordered splits instead, such as walk-forward validation (scikit-learn's TimeSeriesSplit).

What is the process of adding tuning parameters to a model to induce smoothness in order to prevent overfitting?

Regularization: a penalty on parameter size added to the loss, so the fit stays smooth rather than chasing every training point. L1 (lasso) zeroes weights outright and selects features, L2 (ridge) shrinks them all. Its strength is tuned on validation data, never training loss.

What is count encoding?

Count encoding replaces each categorical value with the number of times it appears in the dataset.

What is one hot encoding?

Encoding a categorical feature as one binary column per category: each row gets a 1 in the column for its own category and 0 in all the others, so every encoded vector is sparse with exactly one 1.

What is target encoding?

Target encoding replaces a categorical value with the average value of the target for that value of the feature.

What is data leakage?

When information from outside the training dataset is used to create the model. For example, including any information from the validation or test sets into the model.

Imagine you are applying a decision-tree-based model to a numerical dataset and you run an experiment where you scale the features. Which model would give better results, the original or the scaled one?

They should give the same result because tree-based models are scale invariant.

What is the difference between L1 (Lasso) and L2 (Ridge) regularization?

L1 penalizes the absolute magnitude of the coefficients while L2 penalizes the square of the coefficients.

What is a confusion matrix?

A table that shows how a classifier's predicted classes compare with the actual classes. For binary classification it is a 2x2 grid of true positives, false positives, true negatives and false negatives; with k classes it is a k × k grid with correct predictions on the diagonal.

Should a confusion matrix be used for supervised or unsupervised learning problems?

Supervised, specifically classification: it compares predicted classes with true labels.

What is boosting when referring to machine learning algorithms?

A family of ensemble methods that trains weak learners in sequence, each one focusing on the examples the previous ones got wrong, and combines them in a weighted vote or sum to form a strong learner. AdaBoost and gradient boosting are the best-known examples.

Explain what bucketing is.

Converting a (usually continuous) feature into multiple binary features called buckets or bins, typically based on value range.

How many centroids would a k-means algorithm with k = 3 find?

3

What is the difference between a dense and a sparse feature?

A dense feature is one in which most values are non-zero in contrast with a sparse one where most values are zeros or empty.

What is downsampling in the context of class-imbalanced dataset?

Training on a disproportionately low percentage of over-represented class examples in order to improve model training on under-represented classes.

What is early stopping?

A method for regularization that involves ending model training before training loss finishes decreasing. In early stopping, you end model training when the loss on a validation dataset starts to increase.

Is logistic regression useful for estimating continuous variables?

No. Logistic regression models a categorical, usually binary, outcome: its output is the probability, between 0 and 1, that an example is in the positive class, not an estimate of a continuous target such as a price.

Creating a model that matches the training data so closely that the model fails to make correct predictions on new data.

Overfitting: the model has memorised noise particular to the training set, so training error keeps falling while validation error turns back up. The fixes matter as much as the name: more data, regularization, early stopping, a simpler model, and cross-validation to detect it.

Explain the basic concept of random forest.

An ensemble of many decision trees, each trained on a bootstrap sample of the rows and choosing each split from a random subset of features. It predicts by majority vote or by averaging the trees' outputs; it doesn't pick one best tree.

What does "random" mean in the term random forest?

Two sources of randomness: each tree is trained on a bootstrap sample of the rows, and each split considers only a random subset of the features, drawn afresh at every split. Averaging these decorrelated trees reduces variance.

If someone asks you: out of all the actual positive labels, how many did the model correctly identify? Which metric should you calculate?

Recall: true positives over all actual positives, TP / (TP + FN). It is the metric to optimise when a miss is the expensive error, as in fraud or disease screening. You can always buy recall by lowering the threshold, which is why it is quoted next to precision.

Producing a model with poor predictive ability because the model hasn't captured the complexity of the training data.

Underfitting: high bias, a model too simple for the signal, so training and validation error are both high and close to each other. Add capacity or better features, weaken the regularization, or train longer. More data will not rescue a model that cannot fit what it already has.

What is precision?

Precision is the number of correct positive predictions divided by the total number of positive predictions: TP / (TP + FP).

What is recall?

Recall is the number of the correct predictions of the positive class divided by the number of predictions that should have been classified in the positive class. TP/(TP + FN)

What is F1-score?

Is a measure of a test's accuracy. F1-score is the harmonic mean of the precision and recall.

Is it better to have an F1 score of 1 or of 0?

F1-score reaches its best value at 1 (perfect precision and recall).

Which regularization technique penalizes the sum of absolute value of weights? L1 or L2?

L1.

Which regularization technique penalizes the sum of square weights? L1 or L2?

L2.

L1 regularization is also called?

Lasso, short for least absolute shrinkage and selection operator. It adds λ times the L1 norm of the weights, λ Σ |wᵢ|, to the loss, which drives some weights exactly to zero.

L2 regularization is also called?

Ridge.

Can we have both L1 and L2 regularization components in a linear model?

Yes, it is called Elastic net regularization.

What is Elastic net regularization?

A regularization technique that linearly combines the L1 (lasso) and L2 (ridge) penalties.

What is unsupervised learning?

Unsupervised learning aims to detect patterns in data where no labels are given.

Does feature selection tend to increase overfitting?

No. It actually could help to reduce overfitting.

Mean Squared Error, Root Mean Squared Error, Mean Absolute Error, R-squared are metrics for evaluation which type of models?

Regression models.

Model simplification tends to decrease bias and increase variance, which leads to underfitting. True or False?

False. Model simplification decreases the variance but increases the bias.

Increasing model complexity tends to decrease bias and increase variance, which leads to overfitting. True or False?

True.

Accuracy, Precision, Recall, F1 score, ... are metrics to evaluate which types of models?

Classification models.

Why is naive Bayes 'naive'?

Because it assumes the features are independent of each other given the class, so an example's likelihood is simply the product of each feature's likelihood.

Is KNN a clustering algorithm?

No. It is a supervised learning method that could be used for classification and regression problems.

How is random forest different from gradient boosting algorithm?

The fundamental difference is, random forest uses bagging technique to make predictions. GBM uses boosting techniques to make predictions.

Is it possible to design a linear regression algorithm using a neural network?

Yes.

Which regularization would you use if your model is underfitting?

None: regularization fights overfitting, and adding it to an underfitting model makes things worse. If the model is already regularized, weaken or remove it, and add capacity or better features.

Is standardization of features required before training a logistic regression?

No — rescaling a feature just rescales its coefficient, so the predictions are unchanged. It still matters in practice: gradient descent converges far faster when features share a scale, and any L1 or L2 penalty is scale-dependent, so unscaled features get penalised unevenly.

What is the difference between online and batch learning?

Batch (offline) learning trains on the whole dataset at once and must be retrained to absorb new data. Online learning updates the model incrementally as each new example, or small group of examples, arrives.

Bagging stands for?

Bootstrap aggregating: train the same learner on many bootstrap resamples and average the predictions, which cuts variance and leaves bias roughly untouched. That is why it suits deep unstable trees, while boosting attacks bias by fitting each model to the last one's errors.

What is the difference between boosting and bagging?

Bagging trains models independently on bootstrap samples and averages or votes their predictions, which mainly reduces variance. Boosting trains weak learners one after another, each focusing on the errors of the ensemble so far, and combines them in a weighted sum, which mainly reduces bias.

High bias can cause an algorithm to underfit? True or False?

True.

High variance can cause an algorithm to overfit. True or False?

True.

What is variance in the context of machine learning?

Variance is the error that comes from a model's sensitivity to small fluctuations in the training set: trained on a different sample, a high-variance model makes noticeably different predictions. High variance can cause an algorithm to model the random noise in the training data, which is overfitting.

Explain the bias-variance tradeoff.

Expected squared error splits into squared bias (a model too simple for the truth), variance (sensitivity to which training sample you drew) and irreducible noise. Pushing one down usually lifts the other, so you tune capacity and regularization to minimise the sum, not either term alone.

What happens to our linear regression if we have columns x, y and z, and z is the sum of x and y?

The coefficients are no longer unique: z is a linear combination of x and y (perfect multicollinearity), so XᵀX is singular and infinitely many coefficient sets give the same fit. Predictions still work; drop z or use ridge regression.

Is logistic regression a linear model? Why?

Logistic regression is considered a generalized linear model because the outcome always depends on the sum of the inputs and parameters.

What is TF-IDF?

TF-IDF weights a word in a document by its term frequency (how often it appears there) times its inverse document frequency, log(N / df), which is high for words few documents contain. Words frequent in one document but rare across the corpus score highest.

What is overfitting?

When a model performs very well on its training set but poorly on new data, such as a test set, because it has fit the noise and quirks of the training data instead of only the underlying pattern.

What is the precision-recall curve?

A precision-recall curve (or PR Curve) is a plot of the precision (y-axis) and the recall (x-axis) for different probability thresholds.

Can we use L1 regularization for feature selection?

Yes, because the nature of L1 regularization will lead to sparse coefficients of features. Feature selection can be done by keeping only features with non-zero coefficients.

What is bag of words?

Bag of Words is a representation of text that describes the occurrence of words within a document.

What is clustering?

Clustering algorithms group objects such that similar feature points are put into the same groups (clusters).

What is a time series?

A time series is a set of observations ordered in time usually collected at regular intervals.

When building your vocabulary, is it OK to use the tokens from the entire dataset to shrink your out-of-vocabulary list?

False. Build the vocabulary from the training split only (or from an external corpus, such as a pretrained model's vocabulary); tokens seen only in validation or test data map to UNK, or are split into known subword pieces by a subword tokenizer.

What is the main data structure used in causal modelling?

Graphs! More specifically, directed acyclic graphs.

What is the difference between sensitivity and specificity?

Sensitivity is the share of actual positives a test catches, TP / (TP + FN), the same as recall. Specificity is the share of actual negatives it correctly clears, TN / (TN + FP). Moving the decision threshold trades one for the other.

What does it mean for a classifier to be well calibrated?

Its predicted probabilities match observed frequencies: of all the cases it scores 0.8, about 80% turn out positive. A calibration curve, or reliability diagram, plots the observed positive rate against the predicted probability; a calibrated model follows the diagonal.

What is the Brier score?

The mean squared difference between predicted probabilities and actual outcomes coded as 0 or 1: (1/N) Σ (pᵢ − yᵢ)². Lower is better: 0 is perfect, and always predicting 0.5 scores 0.25.

What is log loss?

The average negative log of the probability the model gave to the true class: −(1/N) Σ [yᵢ log pᵢ + (1 − yᵢ) log(1 − pᵢ)] for binary labels. It rewards confident correct predictions and punishes confident wrong ones heavily.

What is hinge loss?

The loss behind support vector machines: max(0, 1 − y·f(x)) with labels y = ±1 and raw score f(x). It is zero for points on the correct side of the margin and grows linearly as a point moves into the margin or past the boundary.

What is Huber loss?

A regression loss that is squared for small errors and linear for large ones, with a threshold δ between the two. It behaves like squared error near zero and like absolute error for outliers, so a few extreme points can't dominate the fit.

What is pinball loss?

The loss used to predict a quantile instead of a mean: for quantile τ, an under-prediction costs τ × error and an over-prediction costs (1 − τ) × error. Minimizing it with τ = 0.9 gives a forecast that actual values fall below 90% of the time.

What is MAPE?

Mean absolute percentage error: the average of |actual − forecast| / |actual|, as a percentage. It is easy to explain, but it is undefined when an actual value is 0, explodes near 0, and penalizes over-forecasts more than under-forecasts.

What is WAPE?

Weighted absolute percentage error: total absolute error divided by total actual volume, Σ|actual − forecast| / Σ|actual|. Unlike MAPE it doesn't blow up on small or zero actuals, because each error counts in proportion to volume.

What is NDCG?

Normalized discounted cumulative gain, a ranking metric: add up each result's relevance, discounted by the log of its position, then divide by the same sum for the ideal ordering. It runs from 0 to 1 and rewards putting the most relevant items at the top.

What is Cohen's kappa?

A measure of agreement between two raters, or between a classifier and the truth, that corrects for agreement expected by chance: κ = (p_o − p_e) / (1 − p_e). A value of 1 is perfect agreement and 0 is what chance alone would give.

What is the Matthews correlation coefficient?

A single score for a binary classifier that uses all four cells of the confusion matrix: MCC = (TP·TN − FP·FN) / √((TP+FP)(TP+FN)(TN+FP)(TN+FN)). It runs from −1 to 1, with 0 for random guessing, and stays honest on imbalanced data.