Active Subspace of Neural Networks: Structural Analysis and Universal
  Attacks
  Active subspace is a model reduction method widely used in the uncertainty
quantification community. In this paper, we propose analyzing the internal
structure and vulnerability and deep neural networks using active subspace.
Firstly, we employ the active subspace to measure the number of "active
neurons" at each intermediate layer and reduce the number of neurons from
several thousands to several dozens. This motivates us to change the network
structure and to develop a new and more compact network, referred to as
{ASNet}, that has significantly fewer model parameters. Secondly, we propose
analyzing the vulnerability of a neural network using active subspace and
finding an additive universal adversarial attack vector that can misclassify a
dataset with a high probability. Our experiments on CIFAR-10 show that ASNet
can achieve 23.98$\times$ parameter and 7.30$\times$ flops reduction. The
universal active subspace attack vector can achieve around 20% higher attack
ratio compared with the existing approach in all of our numerical experiments.
The PyTorch codes for this paper are available online.
