What is dropout regularization?

Dropout is a regularization technique for neural networks where, during training, each neuron is randomly deactivated (set to zero) with probability p (typically 0.2–0.5) at each forward pass. This prevents neurons from co-adapting too strongly and forces the network to learn redundant representations. It can be interpreted as training an ensemble of 2ⁿ different network architectures simultaneously. At inference time, dropout is disabled and weights are scaled by (1-p). Dropout is one of the most effective regularization techniques for large neural networks, introduced by Hinton et al. (2012).