Deep Global-Connected Net With The Generalized Multi-Piecewise ReLU
  Activation in Deep Learning
  Recent Progress has shown that exploitation of hidden layer neurons in
convolution neural networks incorporating with a carefully designed activation
function can yield better classification results in the field of computer
vision. The paper firstly introduces a novel deep learning architecture aiming
to mitigate the gradient-vanishing problem, in which the earlier hidden layer
neurons could be directly connected with the last hidden layer and feed into
the last layer for classification. We then design a generalized linear
rectifier function as the activation function that can approximate arbitrary
complex functions via training of the parameters. We will show that our design
can achieve similar performance in a number of object recognition and video
action benchmark tasks, under significantly less number of parameters and
shallower network infrastructure, which is not only promising in training in
terms of computation burden and memory usage, but is also applicable to
low-computation, low-memory mobile scenarios.
