SRP: Efficient class-aware embedding learning for large-scale data via
  supervised random projections
  Supervised dimensionality reduction strategies have been of great interest.
However, current supervised dimensionality reduction approaches are difficult
to scale for situations characterized by large datasets given the high
computational complexities associated with such methods. While stochastic
approximation strategies have been explored for unsupervised dimensionality
reduction to tackle this challenge, such approaches are not well-suited for
accelerating computational speed for supervised dimensionality reduction.
Motivated to tackle this challenge, in this study we explore a novel direction
of directly learning optimal class-aware embeddings in a supervised manner via
the notion of supervised random projections (SRP). The key idea behind SRP is
that, rather than performing spectral decomposition (or approximations thereof)
which are computationally prohibitive for large-scale data, we instead perform
a direct decomposition by leveraging kernel approximation theory and the
symmetry of the Hilbert-Schmidt Independence Criterion (HSIC) measure of
dependence between the embedded data and the labels. Experimental results on
five different synthetic and real-world datasets demonstrate that the proposed
SRP strategy for class-aware embedding learning can be very promising in
producing embeddings that are highly competitive with existing supervised
dimensionality reduction methods (e.g., SPCA and KSPCA) while achieving 1-2
orders of magnitude better computational performance. As such, such an
efficient approach to learning embeddings for dimensionality reduction can be a
powerful tool for large-scale data analysis and visualization.
