Partial local entropy and anisotropy in deep weight spaces
  We refine a recently-proposed class of local entropic loss functions by
restricting the smoothening regularization to only a subset of weights. The new
loss functions are referred to as partial local entropies. They can adapt to
the weight-space anisotropy, thus outperforming their isotropic counterparts.
We support the theoretical analysis with experiments on image classification
tasks performed with multi-layer, fully-connected and convolutional neural
networks. The present study suggests how to better exploit the anisotropic
nature of deep landscapes and provides direct probes of the shape of the minima
encountered by stochastic gradient descent algorithms. As a by-product, we
observe an asymptotic dynamical regime at late training times where the
temperature of all the layers obeys a common cooling behavior.
