Boosting Network Weight Separability via Feed-Backward Reconstruction
  This paper proposes a new evaluation metric and boosting method for weight
separability in neural network design. In contrast to general visual
recognition methods designed to encourage both intra-class compactness and
inter-class separability of latent features, we focus on estimating linear
independence of column vectors in weight matrix and improving the separability
of weight vectors. To this end, we propose an evaluation metric for weight
separability based on semi-orthogonality of a matrix and Frobenius distance,
and the feed-backward reconstruction loss which explicitly encourages weight
separability between the column vectors in the weight matrix. The experimental
results on image classification and face recognition demonstrate that the
weight separability boosting via minimization of feed-backward reconstruction
loss can improve the visual recognition performance, hence universally boosting
the performance on various visual recognition tasks.
