Deep Neural Networks for Rank-Consistent Ordinal Regression Based On
  Conditional Probabilities
  In recent times, deep neural networks achieved outstanding predictive
performance on various classification and pattern recognition tasks. However,
many real-world prediction problems have ordinal response variables, and this
ordering information is ignored by conventional classification losses such as
the multi-category cross-entropy. Ordinal regression methods for deep neural
networks address this. One such method is the CORAL method, which is based on
an earlier binary label extension framework and achieves rank consistency among
its output layer tasks by imposing a weight-sharing constraint. However, while
earlier experiments showed that CORAL's rank consistency is beneficial for
performance, the weight-sharing constraint could severely restrict the
expressiveness of a deep neural network. In this paper, we propose an
alternative method for rank-consistent ordinal regression that does not require
a weight-sharing constraint in a neural network's fully connected output layer.
We achieve this rank consistency by a novel training scheme using conditional
training sets to obtain the unconditional rank probabilities through applying
the chain rule for conditional probability distributions. Experiments on
various datasets demonstrate the efficacy of the proposed method to utilize the
ordinal target information, and the absence of the weight-sharing restriction
improves the performance substantially compared to the CORAL reference
approach.
