MMA Regularization: Decorrelating Weights of Neural Networks by
  Maximizing the Minimal Angles
  The strong correlation between neurons or filters can significantly weaken
the generalization ability of neural networks. Inspired by the well-known
Tammes problem, we propose a novel diversity regularization method to address
this issue, which makes the normalized weight vectors of neurons or filters
distributed on a hypersphere as uniformly as possible, through maximizing the
minimal pairwise angles (MMA). This method can easily exert its effect by
plugging the MMA regularization term into the loss function with negligible
computational overhead. The MMA regularization is simple, efficient, and
effective. Therefore, it can be used as a basic regularization method in neural
network training. Extensive experiments demonstrate that MMA regularization is
able to enhance the generalization ability of various modern models and
achieves considerable performance improvements on CIFAR100 and TinyImageNet
datasets. In addition, experiments on face verification show that MMA
regularization is also effective for feature learning. Code is available at:
https://github.com/wznpub/MMA_Regularization.
