Deep Learning
As the number of hidden layers increases in a neural network, its capacity also increases.
True
Proposed by @svpino
Card 1 of 259. Answer: True
True
Proposed by @svpino
Card 1 of 259. Answer: True
About · Quizzes · History of AI · To suggest new questions or report an error send me a dm.
True. A network's capacity is the range of functions it can represent, and adding hidden layers enlarges it: each layer adds parameters and another non-linear stage, so complex functions can be built by composing simpler ones. A new ReLU layer at least as wide as the previous one can copy that layer's non-negative outputs unchanged, so the deeper network loses nothing.
Depth adds capacity faster than width. With one input feature, one hidden layer of n ReLU units gives a piecewise-linear function with at most n + 1 pieces. Ten layers of two units, each folding the interval [0, 1] in half, give 2¹⁰ = 1,024 pieces from 20 units; one hidden layer would need 1,023.
Capacity is what a network can represent, not what it will learn. More of it raises the risk of overfitting, and very deep plain networks are hard to optimize: in the ResNet paper, a 56-layer plain network had higher training error on CIFAR-10 than a 20-layer one. Residual connections were the fix.