Explicitly Bayesian Regularizations in Deep Learning
  Generalization is essential for deep learning. In contrast to previous works
claiming that Deep Neural Networks (DNNs) have an implicit regularization
implemented by the stochastic gradient descent, we demonstrate explicitly
Bayesian regularizations in a specific category of DNNs, i.e., Convolutional
Neural Networks (CNNs). First, we introduce a novel probabilistic
representation for the hidden layers of CNNs and demonstrate that CNNs
correspond to Bayesian networks with the serial connection. Furthermore, we
show that the hidden layers close to the input formulate prior distributions,
thus CNNs have explicitly Bayesian regularizations based on the Bayesian
regularization theory. In addition, we clarify two recently observed empirical
phenomena that are inconsistent with traditional theories of generalization.
Finally, we validate the proposed theory on a synthetic dataset
