Identifying Critical Neurons in ANN Architectures using Mixed Integer
  Programming
  We introduce a mixed integer program (MIP) for assigning importance scores to
each neuron in deep neural network architectures which is guided by the impact
of their simultaneous pruning on the main learning task of the network. By
carefully devising the objective function of the MIP, we drive the solver to
minimize the number of critical neurons (i.e., with high importance score) that
need to be kept for maintaining the overall accuracy of the trained neural
network. Further, the proposed formulation generalizes the recently considered
lottery ticket optimization by identifying multiple "lucky" sub-networks
resulting in optimized architecture that not only performs well on a single
dataset, but also generalizes across multiple ones upon retraining of network
weights. Finally, we present a scalable implementation of our method by
decoupling the importance scores across layers using auxiliary networks. We
demonstrate the ability of our formulation to prune neural networks with
marginal loss in accuracy and generalizability on popular datasets and
architectures.
