Algorithmic Stability and Generalization of an Unsupervised Feature
  Selection Algorithm
  Feature selection, as a vital dimension reduction technique, reduces data
dimension by identifying an essential subset of input features, which can
facilitate interpretable insights into learning and inference processes.
Algorithmic stability is a key characteristic of an algorithm regarding its
sensitivity to perturbations of input samples. In this paper, we propose an
innovative unsupervised feature selection algorithm attaining this stability
with provable guarantees. The architecture of our algorithm consists of a
feature scorer and a feature selector. The scorer trains a neural network (NN)
to globally score all the features, and the selector adopts a dependent sub-NN
to locally evaluate the representation abilities for selecting features.
Further, we present algorithmic stability analysis and show that our algorithm
has a performance guarantee via a generalization error bound. Extensive
experimental results on real-world datasets demonstrate superior generalization
performance of our proposed algorithm to strong baseline methods. Also, the
properties revealed by our theoretical analysis and the stability of our
algorithm-selected features are empirically confirmed.
