Label-similarity Curriculum Learning
  Curriculum learning can improve neural network training by guiding the
optimization to desirable optima. We propose a novel curriculum learning
approach for image classification that adapts the loss function by changing the
label representation. The idea is to use a probability distribution over
classes as target label, where the class probabilities reflect the similarity
to the true class. Gradually, this label representation is shifted towards the
standard one-hot-encoding. That is, in the beginning minor mistakes are
corrected less than large mistakes, resembling a teaching process in which
broad concepts are explained first before subtle differences are taught.
  The class similarity can be based on prior knowledge. For the special case of
the labels being natural words, we propose a generic way to automatically
compute the similarities. The natural words are embedded into Euclidean space
using a standard word embedding. The probability of each class is then a
function of the cosine similarity between the vector representations of the
class and the true label. The proposed label-similarity curriculum learning
(LCL) approach was empirically evaluated using several popular deep learning
architectures for image classification tasks applied to five datasets including
ImageNet, CIFAR100, and AWA2. In all scenarios, LCL was able to improve the
classification accuracy on the test data compared to standard training.
