Automatic Node Selection for Deep Neural Networks using Group Lasso
  Regularization
  We examine the effect of the Group Lasso (gLasso) regularizer in selecting
the salient nodes of Deep Neural Network (DNN) hidden layers by applying a
DNN-HMM hybrid speech recognizer to TED Talks speech data. We test two types of
gLasso regularization, one for outgoing weight vectors and another for incoming
weight vectors, as well as two sizes of DNNs: 2048 hidden layer nodes and 4096
nodes. Furthermore, we compare gLasso and L2 regularizers. Our experiment
results demonstrate that our DNN training, in which the gLasso regularizer was
embedded, successfully selected the hidden layer nodes that are necessary and
sufficient for achieving high classification power.
