Defensive Tensorization
  We propose defensive tensorization, an adversarial defence technique that
leverages a latent high-order factorization of the network. The layers of a
network are first expressed as factorized tensor layers. Tensor dropout is then
applied in the latent subspace, therefore resulting in dense reconstructed
weights, without the sparsity or perturbations typically induced by the
randomization.Our approach can be readily integrated with any arbitrary neural
architecture and combined with techniques like adversarial training. We
empirically demonstrate the effectiveness of our approach on standard image
classification benchmarks. We validate the versatility of our approach across
domains and low-precision architectures by considering an audio classification
task and binary networks. In all cases, we demonstrate improved performance
compared to prior works.
