Machine Learning interview questions
Models, metrics and the tradeoffs that decide whether a model survives contact with real data.
70 questions in this category.
- Name three supervised learning algorithms.
- Support Vector Machines, Regression, Naive Bayes, Decision Trees, Neural Networks, etc.
- Name three unsupervised learning algorithms.
- Clustering, Anomaly Detection, Latent variable models, Autoencoders, etc.
- What is the type of machine learning algorithms used to draw inferences from labeled data?
- Supervised Learning.
- What is the type of machine learning algorithms used to draw inferences from data without labels?
- Unsupervised Learning.
- Name three types of kernels in SVM.
- Linear, polynomial, radical, sigmoid
- What is pruning in a decision tree?
- When we remove the sub-nodes of the decision tree.
- What is ensemble learning?
- Combining individual models together with the purpose of improving the predictive power of the model.
- Would you use k-fold cross validation on time series data? Explain.
- No, you should be aware to the fact that a time series is not randomly distributed data.
- What is the process of adding tuning parameters to a model to induce smoothness in order to prevent overfitting?
- Regularization.
- What is count encoding?
- Count encoding replaces each categorical value with the number of times it appears in the dataset.
- What is one hot encoding?
- A sparse vector in which: One element is set to 1 and all other elements are set to 0.
- What is target encoding?
- Target encoding replaces a categorical value with the average value of the target for that value of the feature.
- What is data leakage?
- When information from outside the training dataset is used to create the model. For example, including any information from the validation or test sets into the model.
- Imagine you are applying a decision tree based model to a numerical dataset and you run a experiment where you scale the features. What model would give the best results the original or the scaleted one?
- They should give the same result because tree-based models are scale invariant.
- What is the difference between L1 (Lasso) and L2 (Ridge) regularization?
- L1 penalizes the absolute magnitude of the coefficients while L2 penalizes the square of the coefficients.
- What is a confusion matrix?
- A confusion matrix lets you see for a given model how your predictions compare with the actual results. It’s a 2x2 grid that has four parts: the number of true positives, false positives, true negatives, and false negatives.
- What is boosting when referring to machine learning algorithms?
- Boosting refers to a whole class of machine learning algorithms that are built on taking a weak model and reusing it enough times so that it becomes a strong one.
- Explain what is bucketing.
- Converting a (usually continuous) feature into multiple binary features called buckets or bins, typically based on value range.
- What is the difference between a dense and a sparse feature?
- A dense feature is one in which most values are non-zero in contrast with a sparse one where most values are zeros or empty.
- What is downsampling in the context of class-imbalanced dataset?
- Training on a disproportionately low percentage of over-represented class examples in order to improve model training on under-represented classes.
- What is early stopping?
- A method for regularization that involves ending model training before training loss finishes decreasing. In early stopping, you end model training when the loss on a validation dataset starts to increase.
- Explain the basic concept of random forest.
- An ensemble approach to finding the decision tree that best fits the training data by creating many decision trees and then determining the "average" one.
- What does random means in the random forest term?
- The "random" part of the term refers to building each of the decision trees from a random selection of features.
- Producing a model with poor predictive ability because the model hasn't captured the complexity of the training data.
- Underfitting.
- What is precision?
- Precision is the number of the correct predictions of the positive class divided by the all the predicted positive class. TP/(TP + FP)
- What is recall?
- Recall is the number of the correct predictions of the positive class divided by the number of predictions that should have been classified in the positive class. TP/(TP + FN)
- What is F1-score?
- Is a measure of a test's accuracy. F1-score is the harmonic mean of the precision and recall.
- What is better to have a F1-score equals to 1 or to 0?
- F1-score reaches its best value at 1 (perfect precision and recall).
- L1 regularization is also called?
- Lasso or L1 norm.
- Can we have both L1 and L2 regularization components in a linear model?
- Yes, it is called Elastic net regularization.
- What is Elastic net regularization?
- It is regularization technique that linearly combines L1 and L2 penalties.
- What is unsupervised learning?
- Unsupervised learning aims to detect patterns in data where no labels are given.
- Does feature selection tends to increase overfitting?
- No. It actually could help to reduce overfitting.
- Mean Squared Error, Root Mean Squared Error, Mean Absolute Error, R-squared are metrics for evaluation which type of models?
- Regression models.
- Model simplification tends to decrease the bias and increase the variance which leads to underfitting. True or False?
- False. Model simplification decreases the variance but increases the bias.
- Accuracy, Precision, Recall, F1 score, ... are metrics to evaluate which types of models?
- Classification models.
- Why is naive Bayes 'naive'?
- Because it assumes that all of the features in a data set are equally important and independent.
- Is KNN a clustering algorithm?
- No. It is and supervised learning method that could be used for classification and regression problems.
- How is random forest different from gradient boosting algorithm?
- The fundamental difference is, random forest uses bagging technique to make predictions. GBM uses boosting techniques to make predictions.
- Which regularization would you use if your model is underfitting?
- None, because regularization is used in case of overfitting.
- What is the difference between online and batch Learning?
- Online: you would use the "most recent" sample at each iteration. Batch: Learning over groups of patterns.
- Bagging stands for?
- Bootstrap Aggregation.
- What is the difference between boosting and bagging?
- In bagging we take boostrap samples of the data (with replacement) and each sample trains a weak learner. And boosting uses all data to train each learner and then average the result using a weighted average approach.
- What is variance in the context of Machine Learning?
- Is a type of error that occurs due to a model's sensitivity to small fluctuations in the training set. High variance can cause an algorithm to model the random noise in the training data.
- Explain what is Bias-Variance tradeoff?
- Usually models with low bias have high variance and vice versa.
- What happens to our linear regression if we have columns x, y, z. And z is a sum of x and y?
- We would not be able to perform the resgression. Beacuse z is linear dependent of x and y.
- Is logistic regression a linear model? Why?
- Logistic regression is considered a generalized linear model because the outcome always depends on the sum of the inputs and parameters.
- What is TF-IDF?
- Term Frequency (TF) is a scoring of the frequency of the word in the current document. Inverse Document Frequency(IDF) is a scoring of how rare the word is across documents.
- What is overfitting?
- When your model perform very well on your training set but can’t generalize the test set, because it adjusted a lot to the training set.
- What is the precision-recall curve?
- A precision-recall curve (or PR Curve) is a plot of the precision (y-axis) and the recall (x-axis) for different probability thresholds.
- Can we use L1 regularization for feature selection?
- Yes, because the nature of L1 regularization will lead to sparse coefficients of features. Feature selection can be done by keeping only features with non-zero coefficients.
- What is bag of words?
- Bag of Words is a representation of text that describes the occurrence of words within a document.
- What is clustering?
- Clustering algorithms group objects such that similar feature points are put into the same groups (clusters).
- What is a time series?
- A time series is a set of observations ordered in time usually collected at regular intervals.
- When building your vocabulary set, it’s ok to use the tokens from the entire dataset to reduce the size of your out-of-vocab list?
- False, any tokens not seen in the training set need to be UNK tokens.
- What is the main data structure used in causal modelling?
- Graphs! More specifically, directed acyclic graphs.