Label Embedding Network: Learning Label Representation for Soft Training
  of Deep Networks
  We propose a method, called Label Embedding Network, which can learn label
representation (label embedding) during the training process of deep networks.
With the proposed method, the label embedding is adaptively and automatically
learned through back propagation. The original one-hot represented loss
function is converted into a new loss function with soft distributions, such
that the originally unrelated labels have continuous interactions with each
other during the training process. As a result, the trained model can achieve
substantially higher accuracy and with faster convergence speed. Experimental
results based on competitive tasks demonstrate the effectiveness of the
proposed method, and the learned label embedding is reasonable and
interpretable. The proposed method achieves comparable or even better results
than the state-of-the-art systems. The source code is available at
\url{https://github.com/lancopku/LabelEmb}.
