Dropout: Explicit Forms and Capacity Control
  We investigate the capacity control provided by dropout in various machine
learning problems. First, we study dropout for matrix completion, where it
induces a data-dependent regularizer that, in expectation, equals the weighted
trace-norm of the product of the factors. In deep learning, we show that the
data-dependent regularizer due to dropout directly controls the Rademacher
complexity of the underlying class of deep neural networks. These developments
enable us to give concrete generalization error bounds for the dropout
algorithm in both matrix completion as well as training deep neural networks.
We evaluate our theoretical findings on real-world datasets, including
MovieLens, MNIST, and Fashion-MNIST.
