Deep Learning interview questions

Neural networks from backpropagation to attention, and the tricks that make them trainable.

54 questions in this category.

Type of backpropagation where we use only a single training example for calculation of the gradient and update parameters.

Stochastic Gradient Descent.

Type of backpropagation where we use the whole dataset to perform a single step.

Batch Gradient Descent.

Type of backpropagation where we use a fixed number of training examples which is lees than the actual dataset to perform a calculation of the gradient

Mini-batch Gradient Descent.

Name three different deep learning frameworks.

TensorFlow, PyTorch, Keras, Caffe, etc.

What is AdaGrad?

A sophisticated gradient descent algorithm that rescales the gradients of each parameter, effectively giving each parameter an independent learning rate.

Which ones are two actors in a convolutional operation?

Convolutional filter and a slice of an input matrix.

How does dropout regularization works in deep learning?

Dropout regularization works by removing a random selection of a fixed number of the units in a network layer for a single gradient step.

What is an epoch?

A full training pass over the entire dataset such that each example has been seen once.

What is a batch?

Number of training samples in 1 Forward/1 Backward pass.

Which are the two components of a Generative Adversarial Network?

A generator and a discriminator.

What is the function of the generator in a GAN?

It creates new examples.

What is the function of the discriminator in a GAN?

Determine whether examples are real or fake.

What does gradient descent tries to minimize?

A loss function.

Which activation function follows these rules: If input is negative or zero, output is 0 and if input is positive, output is equal to input.

Rectified Linear Unit (ReLU).

Explain what a softmax function does?

Provides probabilities for each possible class in a multi-class classification model. The probabilities add up to exactly 1.0

What is learning rate?

Is a tuning parameter in an optimization algorithm that determines the step size at each iteration while moving toward a minimum of a loss function.

What does LSTM means?

Long Short-Term Memory.

Type of layer that tipically has a cell, input gate, output gate and a forget gate?

LSTM layer.

Sigmoid squishes values to be between?

0 and 1

What does GRU means?

Gated Recurrent Unit.

Tanh function squishes values to be between?

-1 and 1

Type of layer that tipically has a reset gate and a output gate?

GRU layer.

What if we set all the weights of a neural network to 0?

The equations of the learning algorithm would fail to make any changes to the network weights, and the model will be stuck.

What happens when the learning rate is too large?

Can accelerate the training. However, it is possible that we “shoot” too far and miss the minimum of the function that we want to optimize.

What happens when the learning rate is too small?

Takes more time to train but it is possible to find a more precise minimum. The downside can be that the solution is stuck in a local minimum.

Why do we need activation functions?

The main idea of using neural networks is to learn complex nonlinear functions. The Nonlinearity comes only with the activation function without them we are just stacking up multiple linear layers.

What are the problems with sigmoid as an activation function?

The output of the sigmoid function for large positive or negative numbers is almost zero. From this comes the problem of vanishing gradient.

What is ReLU? How is it better than sigmoid?

ReLU is an activation function and it solves the problem of vanishing gradient since it doesn't saturates on higher values.

What’s pooling in CNN? Why do we need it?

Pooling is a technique to downsample the feature map. It allows layers which receive relatively undistorted versions of the input to learn low level features such as lines.

Why did the output dimension of all the embedding and sublayers of original Transformer (Attention is All You Need 2017) need to be the same?

Because of the residual connections.

One hot encoding is an excellent solution to transform categorical features with high cardinality.

false

The F1 Score is a metric defined as the harmonic mean of precision, recall, and accuracy.

false

As the number of hidden layers increases in a neural network, its capacity also increases.

true

Initializing a neural network’s weights with zero is our best bet to allow the network to converge.

false

As the dropout ratio used in a neural network increases, the network’s capacity also increases.

false

Stochastic Gradient Descent is the proper technique to use when the full dataset doesn’t fit in memory.

true

Batch Normalization is an efficient backpropagation technique that allows neural networks to learn.

false

Gradient Descent is an algorithm used to minimize overfitting in neural networks.

false

Having high bias means the model is too simple and can’t capture many features during the training phase. This is also known as overfitting.

false

Precision is a performance metric that defines a classification models ability to identify only relevant samples.

True

Recall is a performance metric that defines a classification models ability to identify all relevant samples.

True

Softmax is an activation function that always returns the input value if it’s positive and zeroes otherwise.

false

In neural networks, an activation function’s most essential role is to decide whether a unit or neuron should fire.

true

Using a learning rate that’s too low will cause the training process to be very slow.

true

ReLU is an activation function that, given an input vector, generates an output where the sum of the values in the vector is equal to one.

false

An autoencoder is a neural network that automatically learns by using its inputs as the expected output.

true

Convolutional neural networks are translation invariant.

true

Autoencoders are one of the most popular supervised learning methods.

false

The type of problem where you need to classify the input into a single class is called multi class classification.

true

Dropout is an excellent regularization technique that speeds the training process.

false

A neural network with a single layer is capable of approximating any function.

true

Leaky ReLU is an activation function that allows the passing of small sized negative values if the network’s input value is less than zero.

true

Bagging is the concept of splitting a dataset and randomly placing it into bags for training a model.

true

A Categorical feature is a feature that can only take a limited number of possible values.

True