Deep Learning
As the dropout ratio used in a neural network increases, the network’s capacity also increases.
False
Proposed by @svpino
Card 1 of 276. Answer: False
False
Proposed by @svpino
Card 1 of 276. Answer: False
About · Quizzes · History of AI · To suggest new questions or report an error send me a dm.
False. A higher dropout rate lowers a network's effective capacity, its ability to fit complex functions, noise included. The rate is the probability that each unit is zeroed on a training step, so a higher rate means each step trains a smaller random sub-network. No parameters are added or removed; dropout only limits how existing weights are used, which is why it regularizes.
Example: in a layer of 1,024 units, a rate of 0.2 leaves about 819 active per step and a rate of 0.8 about 205. Units cannot rely on particular partners and must learn simple, redundant features. Too high a rate underfits: even the training loss stays high.
If strong dropout hurts the fit, widen the layer: Srivastava et al. (2014) suggest that a layer needing n units without dropout get at least n/p with it, where p is the keep probability. That convention is the trap here: Keras and PyTorch take the drop probability, but if a source quotes a keep probability, raising it raises capacity.