Binary Stochastic Filtering: a Method for Neural Network Size
  Minimization and Supervised Feature Selection
  Binary Stochastic Filtering (BSF), the algorithm for feature selection and
neuron pruning is proposed in this work. The method defines filtering layer
which penalizes amount of the information involved in the training process.
This information could be the input data or output of the previous layer, which
directly leads to the feature selection or neuron pruning respectively,
producing \textit{ad hoc} subset of features or selecting optimal number of
neurons in each layer. Filtering layer stochastically passes or drops features
based on individual weights, which are tuned with standard backpropagation
algorithm during the training process. Multifold decrease of neural network
size has been achieved in the experiments. Besides, the method was able to
select minimal number of features, surpassing literature references by the
accuracy/dimensionality ratio.
