Nonlinear Tensor Ring Network
  The state-of-the-art deep neural networks (DNNs) have been widely applied for
various real-world applications, and achieved significant performance for
cognitive problems. However, the increment of DNNs' width and depth in
architecture results in a huge amount of parameters to challenge the storage
and memory cost, limiting to the usage of DNNs on resource-constrained
platforms, such as portable devices. By converting redundant models into
compact ones, compression technique appears to be a practical solution to
reducing the storage and memory consumption. In this paper, we develop a
nonlinear tensor ring network (NTRN) in which both fullyconnected and
convolutional layers are compressed via tensor ring decomposition. Furthermore,
to mitigate the accuracy loss caused by compression, a nonlinear activation
function is embedded into the tensor contraction and convolution operations
inside the compressed layer. Experimental results demonstrate the effectiveness
and superiority of the proposed NTRN for image classification using two basic
neural networks, LeNet-5 and VGG-11 on three datasets, viz. MNIST, Fashion
MNIST and Cifar-10.
