Weight and Gradient Centralization in Deep Neural Networks
  Batch normalization is currently the most widely used variant of internal
normalization for deep neural networks. Additional work has shown that the
normalization of weights and additional conditioning as well as the
normalization of gradients further improve the generalization. In this work, we
combine several of these methods and thereby increase the generalization of the
networks. The advantage of the newer methods compared to the batch
normalization is not only increased generalization, but also that these methods
only have to be applied during training and, therefore, do not influence the
running time during use. Link to CUDA code
https://atreus.informatik.uni-tuebingen.de/seafile/d/8e2ab8c3fdd444e1a135/
