Sum of Ranked Range Loss for Supervised Learning
  In forming learning objectives, one oftentimes needs to aggregate a set of
individual values to a single output. Such cases occur in the aggregate loss,
which combines individual losses of a learning model over each training sample,
and in the individual loss for multi-label learning, which combines prediction
scores over all class labels. In this work, we introduce the sum of ranked
range (SoRR) as a general approach to form learning objectives. A ranked range
is a consecutive sequence of sorted values of a set of real numbers. The
minimization of SoRR is solved with the difference of convex algorithm (DCA).
We explore two applications in machine learning of the minimization of the SoRR
framework, namely the AoRR aggregate loss for binary/multi-class classification
at the sample level and the TKML individual loss for multi-label/multi-class
classification at the label level. A combination loss of AoRR and TKML is
proposed as a new learning objective for improving the robustness of
multi-label learning in the face of outliers in sample and labels alike. Our
empirical results highlight the effectiveness of the proposed optimization
frameworks and demonstrate the applicability of proposed losses using synthetic
and real data sets.
