Neural Networks in Adversarial Setting and Ill-Conditioned Weight Space
  Recently, Neural networks have seen a huge surge in its adoption due to their
ability to provide high accuracy on various tasks. On the other hand, the
existence of adversarial examples have raised suspicions regarding the
generalization capabilities of neural networks. In this work, we focus on the
weight matrix learnt by the neural networks and hypothesize that ill
conditioned weight matrix is one of the contributing factors in neural
network's susceptibility towards adversarial examples. For ensuring that the
learnt weight matrix's condition number remains sufficiently low, we suggest
using orthogonal regularizer. We show that this indeed helps in increasing the
adversarial accuracy on MNIST and F-MNIST datasets.
