How to Train Unstable Looped Tensor Network
  A rising problem in the compression of Deep Neural Networks is how to reduce
the number of parameters in convolutional kernels and the complexity of these
layers by low-rank tensor approximation. Canonical polyadic tensor
decomposition (CPD) and Tucker tensor decomposition (TKD) are two solutions to
this problem and provide promising results. However, CPD often fails due to
degeneracy, making the networks unstable and hard to fine-tune. TKD does not
provide much compression if the core tensor is big. This motivates using a
hybrid model of CPD and TKD, a decomposition with multiple Tucker models with
small core tensor, known as block term decomposition (BTD). This paper proposes
a more compact model that further compresses the BTD by enforcing core tensors
in BTD identical. We establish a link between the BTD with shared parameters
and a looped chain tensor network (TC). Unfortunately, such strongly
constrained tensor networks (with loop) encounter severe numerical instability,
as proved by y (Landsberg, 2012) and (Handschuh, 2015a). We study perturbation
of chain tensor networks, provide interpretation of instability in TC,
demonstrate the problem. We propose novel methods to gain the stability of the
decomposition results, keep the network robust and attain better approximation.
Experimental results will confirm the superiority of the proposed methods in
compression of well-known CNNs, and TC decomposition under challenging
scenarios
