Reducing Data Complexity using Autoencoders with Class-informed Loss
  Functions
  Available data in machine learning applications is becoming increasingly
complex, due to higher dimensionality and difficult classes. There exists a
wide variety of approaches to measuring complexity of labeled data, according
to class overlap, separability or boundary shapes, as well as group morphology.
Many techniques can transform the data in order to find better features, but
few focus on specifically reducing data complexity. Most data transformation
methods mainly treat the dimensionality aspect, leaving aside the available
information within class labels which can be useful when classes are somehow
complex.
  This paper proposes an autoencoder-based approach to complexity reduction,
using class labels in order to inform the loss function about the adequacy of
the generated variables. This leads to three different new feature learners,
Scorer, Skaler and Slicer. They are based on Fisher's discriminant ratio, the
Kullback-Leibler divergence and least-squares support vector machines,
respectively. They can be applied as a preprocessing stage for a binary
classification problem. A thorough experimentation across a collection of 27
datasets and a range of complexity and classification metrics shows that
class-informed autoencoders perform better than 4 other popular unsupervised
feature extraction techniques, especially when the final objective is using the
data for a classification task.
