Data science interview questions and answers
259 questions that come up in data science interviews, grouped by topic. Machine learning and statistics are tested almost everywhere. Deep learning, LLMs and MLOps come up more for ML engineer and research roles, and the history of AI is there for the small talk. The deck covers concepts, not SQL or coding rounds.
Each topic opens with the questions to answer first, with their short answers. Every question links to its own page with a longer explanation, usually with a worked example. To test yourself instead, play the flashcards or take a ten-question quiz.
Machine Learning82 questions
Models, metrics and the tradeoffs that decide whether a model survives contact with real data.
- What is the bias-variance tradeoff?
- Expected squared error splits into squared bias (a model too simple for the truth), variance (sensitivity to which training sample you drew) and irreducible noise. Pushing one down usually lifts the other, so you tune capacity and regularization to minimise the sum, not either term alone.
- What is overfitting?
- When a model performs very well on its training set but poorly on new data, such as a test set, because it has fit the noise and quirks of the training data instead of only the underlying pattern.
- What is the difference between L1 (Lasso) and L2 (Ridge) regularization?
- L1 penalizes the absolute magnitude of the coefficients while L2 penalizes the square of the coefficients.
- What is the difference between boosting and bagging?
- Bagging trains models independently on bootstrap samples and averages or votes their predictions, which mainly reduces variance. Boosting trains weak learners one after another, each focusing on the errors of the ensemble so far, and combines them in a weighted sum, which mainly reduces bias.
- What is a confusion matrix?
- A table that shows how a classifier's predicted classes compare with the actual classes. For binary classification it is a 2x2 grid of true positives, false positives, true negatives and false negatives; with k classes it is a k × k grid with correct predictions on the diagonal.
More machine learning questions
- What are some examples of supervised learning algorithms?
- What are some examples of unsupervised learning algorithms?
- What is the type of machine learning algorithms used to draw inferences from labeled data?
- What is the type of machine learning algorithms used to draw inferences from data without labels?
- What is a ROC curve?
- What are the types of kernels in SVM?
- What is pruning in a decision tree?
- What is ensemble learning?
- Can you use k-fold cross-validation on time series data?
- What is the process of adding tuning parameters to a model to induce smoothness in order to prevent overfitting?
- What is count encoding?
- What is one hot encoding?
- What is target encoding?
- What is data leakage?
- Imagine you are applying a decision-tree-based model to a numerical dataset and you run an experiment where you scale the features. Which model would give better results, the original or the scaled one?
- Should a confusion matrix be used for supervised or unsupervised learning problems?
- What is boosting when referring to machine learning algorithms?
- What is bucketing in machine learning?
- How many centroids would a k-means algorithm with k = 3 find?
- What is the difference between a dense and a sparse feature?
- What is downsampling in the context of class-imbalanced dataset?
- What is early stopping?
- Is logistic regression useful for estimating continuous variables?
- What is it called when a model fits the training data too closely?
- How does a random forest work?
- What does "random" mean in the term random forest?
- If someone asks you: out of all the actual positive labels, how many did the model correctly identify? Which metric should you calculate?
- What is underfitting in machine learning?
- What is precision?
- What is recall?
- What is F1-score?
- Is it better to have an F1 score of 1 or of 0?
- Which regularization technique penalizes the sum of absolute value of weights? L1 or L2?
- Which regularization technique penalizes the sum of square weights? L1 or L2?
- L1 regularization is also called?
- L2 regularization is also called?
- Can we have both L1 and L2 regularization components in a linear model?
- What is Elastic net regularization?
- What is unsupervised learning?
- Does feature selection tend to increase overfitting?
- Mean Squared Error, Root Mean Squared Error, Mean Absolute Error, R-squared are metrics for evaluation which type of models?
- Model simplification tends to decrease bias and increase variance, which leads to underfitting. True or False?
- Increasing model complexity tends to decrease bias and increase variance, which leads to overfitting. True or False?
- Accuracy, Precision, Recall, F1 score, ... are metrics to evaluate which types of models?
- Why is naive Bayes 'naive'?
- Is KNN a clustering algorithm?
- How is random forest different from gradient boosting algorithm?
- Is it possible to design a linear regression algorithm using a neural network?
- Which regularization would you use if your model is underfitting?
- Is standardization of features required before training a logistic regression?
- What is the difference between online and batch learning?
- Bagging stands for?
- High bias can cause an algorithm to underfit? True or False?
- High variance can cause an algorithm to overfit. True or False?
- What is variance in the context of machine learning?
- What happens to our linear regression if we have columns x, y and z, and z is the sum of x and y?
- Is logistic regression a linear model? Why?
- What is TF-IDF?
- What is the precision-recall curve?
- Can we use L1 regularization for feature selection?
- What is bag of words?
- What is clustering?
- What is a time series?
- When building your vocabulary, is it OK to use the tokens from the entire dataset to shrink your out-of-vocabulary list?
- What is the main data structure used in causal modelling?
- What is the difference between sensitivity and specificity?
- What is a calibration curve in machine learning?
- What is the Brier score?
- What is log loss (cross-entropy loss)?
- What is hinge loss?
- What is Huber loss?
- What is pinball loss (quantile loss)?
- What is MAPE?
- What is WAPE?
- What is NDCG?
- What is Cohen's kappa?
- What is the Matthews correlation coefficient?
Statistics56 questions
Inference, uncertainty and the ideas that people most often get subtly wrong in interviews.
- What is a p-value?
- The probability, if the null hypothesis were true, of getting a result at least as extreme as the one observed. A small p-value means the data would be surprising if there were no effect; it is not the probability that the null hypothesis is true.
- What is a confidence interval?
- A range computed from a sample by a method that, over repeated samples, captures the true parameter a stated share of the time, such as 95%. The 95% describes the method, not the chance that this particular interval contains the true value.
- What is a Type I and a Type II error?
- Type I errors are false positives (rejecting a true null hypothesis) and Type II errors are false negatives (missing a real effect).
- What is the central limit theorem?
- The Central Limit Theorem: the mean of n independent draws from a distribution with finite variance is approximately normal for large n, whatever the population's shape. That is what justifies normal-based confidence intervals and t-tests on non-normal data, provided the sample is large enough for the data's skew.
- Why doesn't correlation imply causation?
- Because an association can come from a confounder that drives both variables, from causation running the other way, from how the data were selected, or from chance. Only an experiment, or careful causal analysis of observational data, shows that changing one changes the other.
More statistics questions
- What is the generalization of the median?
- What is the difference between "long" and "wide" format data?
- What is cluster sampling?
- What two parameters define a normal distribution?
- In a normal distribution which parameter indicates where the bell is centered?
- In a normal distribution, which parameter indicates how wide the bell is?
- What is the difference between a nominal and an ordinal feature?
- What does high and low cardinality mean?
- What is R-squared?
- What is a residual?
- What is dimensionality reduction?
- What is a false negative?
- What is a false positive?
- Values distant from most other values are?
- Precision is the ratio between which two quantities?
- What does stationarity in a dataset mean?
- What are the common dimensionality reduction techniques?
- What measures how far a set of numbers is spread out from their average?
- What values could you infer from a boxplot?
- Is rotation necessary in PCA?
- What is the difference between covariance and correlation?
- Is it possible to capture the correlation between a continuous and a categorical variable?
- If pearson correlation is 0 between two variables can we assume that there isn't any relation between them?
- Does mean imputation take feature correlation into account?
- How do you handle missing values in a dataset?
- What is the Law of Large Numbers?
- What is Survivorship bias?
- What is a confounding variable?
- How is the interquartile range calculated?
- What is autocorrelation?
- What is MSE?
- What is RMSE?
- How can we select K for K-means?
- What is precision and recall at k?
- What is KL divergence?
- What is the Jensen-Shannon divergence?
- What is the Kolmogorov-Smirnov test?
- What is heteroscedasticity?
- What is the Poisson distribution?
- What is the binomial distribution?
- What is Simpson's paradox?
- What do skewness and kurtosis measure?
- What is selection bias?
- What is a probability density function?
- What is maximum likelihood estimation?
- What is statistical power?
- What is a t-test?
- What is the Bonferroni correction?
- What is multicollinearity?
- Bayesian vs frequentist statistics: what is the difference?
- What is adjusted R-squared?
Deep Learning56 questions
Neural networks from backpropagation to attention, and the tricks that make them trainable.
- Why do we need activation functions?
- Without a nonlinear activation, stacked layers collapse into one linear map: W₂(W₁x + b₁) + b₂ is just another linear function of x. Activation functions add the nonlinearity that lets a neural network learn complex functions, such as XOR, that no linear model can represent.
- What is ReLU? How is it better than sigmoid?
- ReLU, max(0, x), zeroes negative inputs and passes positive ones through. Unlike sigmoid it doesn't saturate for positive inputs, where its gradient is 1, so it greatly reduces vanishing gradients in deep networks, and it is cheaper to compute.
- What is learning rate?
- Is a tuning parameter in an optimization algorithm that determines the step size at each iteration while moving toward a minimum of a loss function.
- How does dropout regularization work in deep learning?
- During training, each unit in a layer is set to zero independently with probability p on every forward pass, so each gradient step trains a different random sub-network. At inference no units are dropped.
- What’s pooling in CNN? Why do we need it?
- Pooling downsamples each feature map by summarizing small windows, usually with the max or the average. It reduces spatial size and computation, lets deeper layers cover a larger part of the input, and makes features somewhat robust to small shifts. It has no learned weights.
More deep learning questions
- What is stochastic gradient descent?
- What is batch gradient descent?
- What is mini-batch gradient descent?
- What are some popular deep learning frameworks?
- What is AdaGrad?
- What are the two actors in a convolution operation?
- What is an epoch?
- What is a batch?
- Which are the two components of a Generative Adversarial Network?
- What is the function of the generator in a GAN?
- What is the function of the discriminator in a GAN?
- What does gradient descent try to minimize?
- Which activation function outputs max(0, x)?
- What does the softmax function do?
- What does LSTM mean?
- What type of layer typically has a cell, an input gate, an output gate and a forget gate?
- Sigmoid squishes values to be between?
- What does GRU mean?
- Tanh function squishes values to be between?
- Type of layer that typically has a reset gate and an update gate?
- What if we set all the weights of a neural network to 0?
- What happens when the learning rate is too large?
- What happens when the learning rate is too small?
- What are the problems with sigmoid as an activation function?
- Why did the embedding layers and all sub-layers of the original Transformer (Attention Is All You Need, 2017) need the same output dimension?
- Is one-hot encoding good for high-cardinality categorical features?
- Is the F1 score the harmonic mean of precision, recall and accuracy?
- Does adding hidden layers increase a neural network's capacity?
- Should you initialize neural network weights to zero?
- Does a higher dropout rate increase a neural network's capacity?
- Should you use SGD when the dataset doesn't fit in memory?
- Is batch normalization a backpropagation technique?
- Is gradient descent used to reduce overfitting?
- Is high bias the same as overfitting?
- Does precision measure a model's ability to find only relevant samples?
- Does recall measure a model's ability to find all relevant samples?
- Does softmax return the input if it's positive and zero otherwise?
- Is an activation function's main role to decide whether a neuron fires?
- Does a learning rate that is too low slow down training?
- Does ReLU output values that sum to one?
- Does an autoencoder learn by using its inputs as the target?
- Are convolutional neural networks translation invariant?
- Are autoencoders a supervised learning method?
- Is giving each input exactly one class called multi-class classification?
- Does dropout speed up neural network training?
- Can a neural network with a single layer approximate any function?
- Does leaky ReLU let small negative values pass through?
- Does bagging mean splitting a dataset into random bags?
- Is a categorical feature one that takes a limited set of values?
- What is intersection over union (IoU)?
- What is focal loss?
Generative AI16 questions
Large language models in practice: prompting, retrieval, evaluation and the tradeoffs behind shipping them.
- When do you use RAG instead of fine-tuning?
- RAG when the model lacks knowledge: facts that change, are private, or need a citation. Fine-tuning when it lacks behaviour: a format, a tone, a narrow task. An index is cheaper to update than weights, so reach for RAG first.
- What do temperature, top-p and top-k control?
- How the next token is sampled: temperature flattens or sharpens the distribution, top-k keeps the k likeliest tokens, top-p the smallest set whose mass reaches p. Near zero for extraction and code; raise it only when you want variety.
- What is an embedding?
- A fixed-length vector for text, an image or audio, placed so similar meanings land near each other, which turns search into a nearest-neighbour lookup. Vectors only compare within one model, and closeness is not relevance, hence rerankers.
- How do you evaluate a RAG system?
- Retrieval and generation separately. Retrieval: recall@k and nDCG on labelled question-document pairs. Generation: faithfulness (every claim supported by the context), relevance and citations. Many bad answers are retrieval failures.
- What is the KV cache, and why does it cap concurrency?
- Attention keys and values for the prefix, cached per layer, so each new token attends over saved state instead of recomputing the sequence. The cache grows with batch size, sequence length and layers, so GPU memory rather than compute limits throughput.
More generative AI questions
- What are the tradeoffs in RAG chunk size?
- Why combine embeddings with keyword search?
- What actually reduces hallucinations?
- Why is a bigger context window not always the answer?
- Prompt, system prompt, tools: what does each do?
- Why did transformers replace RNNs for most NLP tasks?
- How does BERT differ from a GPT-style model?
- When should you build a workflow instead of an agent?
- When do few-shot examples beat zero-shot prompting?
- What does RLHF change about a base model?
- How does byte pair encoding tokenization work?
MLOps14 questions
What happens after a model ships: drift, monitoring, deployment and the system design questions around them.
- What is data drift?
- A change in the distribution of a model's inputs between training and production, so the live features stop looking like the data it learned from. When P(X) moves but P(y | X) stays the same, it is called covariate shift.
- What is concept drift?
- A change in the relationship between the inputs and the target, P(y | X), so the same inputs now call for a different prediction. The model's learned rules go stale even when the input data looks unchanged.
- What is training-serving skew?
- A mismatch between how features are computed for training and how they are computed when the model serves predictions, so the live model sees inputs that differ from what it learned from. It is a pipeline bug, not a change in the world.
- How do you monitor a model before its labels arrive?
- Watch proxies until the ground truth lands: data quality, drift in the inputs and in the predictions, and, for a well-calibrated classifier, an estimate of performance from its own confidence scores. Confirm with realized performance once the labels arrive.
- Why can a model that wins offline lose in an A/B test?
- Offline metrics are computed on logged data that the old system shaped, against a proxy target, while the A/B test measures the real outcome on live users. Selection bias in the logs, a weak proxy, feedback loops and added latency all open a gap.
More MLOps questions
- What is model drift?
- What is the Population Stability Index (PSI)?
- What is the difference between batch and online inference?
- What is a feature store?
- What is shadow deployment?
- What is a canary deployment?
- What is the cold start problem?
- What is a two-tower model?
- How do you decide when to retrain a model?
History35 questions
The people, papers and moments that built the field, from the perceptron to the Transformer.
- Who invented the perceptron?
- Frank Rosenblatt. Bonus. The invention of the perceptron generated a great deal of excitement and was widely covered in the media.
- When was the Turing Test created?
- 1950, in Turing's paper "Computing Machinery and Intelligence", which swapped the question of whether machines think for an imitation game judged over text. Its weakness is that it rewards convincing conversation rather than understanding — a critique now aimed straight at LLMs.
- Who envisioned and created the ImageNet dataset?
- Fei-Fei Li, who argued from 2007 that models were starved of data rather than ideas and used crowdsourced labelling to reach roughly 14 million images across 20,000+ categories. The 2012 ILSVRC contest run on it is where AlexNet made deep learning mainstream.
- Which computer scientists received the Turing Award for their contributions to deep learning?
- Yoshua Bengio, Geoffrey Hinton, and Yann LeCun.
- What is the name of the computer program that first beat a professional Go player without a handicap?
- AlphaGo, by Google DeepMind. It beat European champion Fan Hui 5-0 in October 2015, the first win by a computer over a professional on a full 19 × 19 board without handicap.
More history questions
- Who is known as the creator of the GAN architecture?
- LeCun, Cortes, and Burges compiled a very famous dataset with 60,000 training images of handwritten digits. What is the name of this dataset?
- The least squares method was first published by which famous French mathematician?
- In 1812 Laplace published "Théorie analytique des probabilités", in which he stated a famous theorem. Which theorem was it?
- Andrey Markov first described the techniques he used to analyse a poem, and these techniques later became known as Markov chains. True or false?
- In 1951 Marvin Minsky and Dean Edmonds conceived what is known as the first neural network machine. What is its name?
- What did Seppo Linnainmaa's 1970 master's thesis introduce?
- In which decade did Tin Kam Ho publish a paper describing random decision forests?
- What did Corinna Cortes and Vladimir Vapnik publish in 1995?
- What was the name of the first computer to beat a reigning world chess champion?
- Who invented the LSTM architecture in deep learning?
- When was Torch released?
- When was the nearest neighbor algorithm created?
- Who coined the term deep learning in 2006?
- When was TensorFlow launched?
- Who is the creator of Keras?
- PyTorch was primarily developed by which company?
- Was the R language designed by statisticians?
- Who are credited with the creation of R?
- What did Christopher Watkins introduce in his 1989 PhD thesis?
- Who is known as the founding father of convolutional nets?
- Who are known as the "Godfathers of AI"?
- Was Bayes's Theorem published after his death?
- What object did Yann LeCun use in his NIPS 2016 keynote to describe the 'layers' of intelligence?
- What defunct deep learning library was started in 2007 by the Montreal lab now known as Mila?
- What language was the original AlexNet (ILSVRC 2012) written in?
- What was the final score of the 2016 match between Google DeepMind’s AlphaGo and Go grandmaster Lee Sedol?
- Whose notable students include Ruslan Salakhutdinov, Ilya Sutskever and Yann LeCun?
- What was the output dimension of the embedding layers and sub-layers of the original Transformer (Attention Is All You Need, 2017)?
- In 2016 Google’s AlphaGo won four out of five games of Go against Lee Sedol, a professional Go player from South Korea (9 dan rank). What hardware powered the AlphaGo software to victory?
About · Quizzes · History of AI · To suggest new questions or report an error send me a dm.