Online Metric Learning for Multi-Label Classification
  Existing research into online multi-label classification, such as online
sequential multi-label extreme learning machine (OSML-ELM) and stochastic
gradient descent (SGD), has achieved promising performance. However, these
works do not take label dependencies into consideration and lack a theoretical
analysis of loss functions. Accordingly, we propose a novel online metric
learning paradigm for multi-label classification to fill the current research
gap. Generally, we first propose a new metric for multi-label classification
which is based on $k$-Nearest Neighbour ($k$NN) and combined with large margin
principle. Then, we adapt it to the online settting to derive our model which
deals with massive volume ofstreaming data at a higher speed online.
Specifically, in order to learn the new $k$NN-based metric, we first project
instances in the training dataset into the label space, which make it possible
for the comparisons of instances and labels in the same dimension. After that,
we project both of them into a new lower dimension space simultaneously, which
enables us to extract the structure of dependencies between instances and
labels. Finally, we leverage the large margin and $k$NN principle to learn the
metric with an efficient optimization algorithm. Moreover, we provide
theoretical analysis on the upper bound of the cumulative loss for our method.
Comprehensive experiments on a number of benchmark multi-label datasets
validate our theoretical approach and illustrate that our proposed online
metric learning (OML) algorithm outperforms state-of-the-art methods.
