Relative Pairwise Relationship Constrained Non-negative Matrix
  Factorisation
  Non-negative Matrix Factorisation (NMF) has been extensively used in machine
learning and data analytics applications. Most existing variations of NMF only
consider how each row/column vector of factorised matrices should be shaped,
and ignore the relationship among pairwise rows or columns. In many cases, such
pairwise relationship enables better factorisation, for example, image
clustering and recommender systems. In this paper, we propose an algorithm
named, Relative Pairwise Relationship constrained Non-negative Matrix
Factorisation (RPR-NMF), which places constraints over relative pairwise
distances amongst features by imposing penalties in a triplet form. Two
distance measures, squared Euclidean distance and Symmetric divergence, are
used, and exponential and hinge loss penalties are adopted for the two measures
respectively. It is well known that the so-called "multiplicative update rules"
result in a much faster convergence than gradient descend for matrix
factorisation. However, applying such update rules to RPR-NMF and also proving
its convergence is not straightforward. Thus, we use reasonable approximations
to relax the complexity brought by the penalties, which are practically
verified. Experiments on both synthetic datasets and real datasets demonstrate
that our algorithms have advantages on gaining close approximation, satisfying
a high proportion of expected constraints, and achieving superior performance
compared with other algorithms.
