Entropic alternatives to initialization
  Local entropic loss functions provide a versatile framework to define
architecture-aware regularization procedures. Besides the possibility of being
anisotropic in the synaptic space, the local entropic smoothening of the loss
function can vary during training, thus yielding a tunable model complexity. A
scoping protocol where the regularization is strong in the early-stage of the
training and then fades progressively away constitutes an alternative to
standard initialization procedures for deep convolutional neural networks,
nonetheless, it has wider applicability. We analyze anisotropic, local entropic
smoothenings in the language of statistical physics and information theory,
providing insight into both their interpretation and workings. We comment some
aspects related to the physics of renormalization and the spacetime structure
of convolutional networks.
