On the Implicit Bias of Dropout
  Algorithmic approaches endow deep learning systems with implicit bias that
helps them generalize even in over-parametrized settings. In this paper, we
focus on understanding such a bias induced in learning through dropout, a
popular technique to avoid overfitting in deep learning. For single
hidden-layer linear neural networks, we show that dropout tends to make the
norm of incoming/outgoing weight vectors of all the hidden nodes equal. In
addition, we provide a complete characterization of the optimization landscape
induced by dropout.
