Improved graph-based SFA: Information preservation complements the
  slowness principle
  Slow feature analysis (SFA) is an unsupervised-learning algorithm that
extracts slowly varying features from a multi-dimensional time series. A
supervised extension to SFA for classification and regression is graph-based
SFA (GSFA). GSFA is based on the preservation of similarities, which are
specified by a graph structure derived from the labels. It has been shown that
hierarchical GSFA (HGSFA) allows learning from images and other
high-dimensional data. The feature space spanned by HGSFA is complex due to the
composition of the nonlinearities of the nodes in the network. However, we show
that the network discards useful information prematurely before it reaches
higher nodes, resulting in suboptimal global slowness and an under-exploited
feature space.
  To counteract these problems, we propose an extension called hierarchical
information-preserving GSFA (HiGSFA), where information preservation
complements the slowness-maximization goal. We build a 10-layer HiGSFA network
to estimate human age from facial photographs of the MORPH-II database,
achieving a mean absolute error of 3.50 years, improving the state-of-the-art
performance. HiGSFA and HGSFA support multiple-labels and offer a rich feature
space, feed-forward training, and linear complexity in the number of samples
and dimensions. Furthermore, HiGSFA outperforms HGSFA in terms of feature
slowness, estimation accuracy and input reconstruction, giving rise to a
promising hierarchical supervised-learning approach.
