Privileged Information for Data Clustering
  Many machine learning algorithms assume that all input samples are
independently and identically distributed from some common distribution on
either the input space X, in the case of unsupervised learning, or the input
and output space X x Y in the case of supervised and semi-supervised learning.
In the last number of years the relaxation of this assumption has been explored
and the importance of incorporation of additional information within machine
learning algorithms became more apparent. Traditionally such fusion of
information was the domain of semi-supervised learning. More recently the
inclusion of knowledge from separate hypothetical spaces has been proposed by
Vapnik as part of the supervised setting. In this work we are interested in
exploring Vapnik's idea of master-class learning and the associated learning
using privileged information, however within the unsupervised setting. Adoption
of the advanced supervised learning paradigm for the unsupervised setting
instigates investigation into the difference between privileged and technical
data. By means of our proposed aRi-MAX method stability of the KMeans algorithm
is improved and identification of the best clustering solution is achieved on
an artificial dataset. Subsequently an information theoretic dot product based
algorithm called P-Dot is proposed. This method has the ability to utilize a
wide variety of clustering techniques, individually or in combination, while
fusing privileged and technical data for improved clustering. Application of
the P-Dot method to the task of digit recognition confirms our findings in a
real-world scenario.
