Analyze and Design Network Architectures by Recursion Formulas
  The effectiveness of shortcut/skip-connection has been widely verified, which
inspires massive explorations on neural architecture design. This work attempts
to find an effective way to design new network architectures. It is discovered
that the main difference between network architectures can be reflected in
their recursion formulas. Based on this, a methodology is proposed to design
novel network architectures from the perspective of mathematical formulas.
Afterwards, a case study is provided to generate an improved architecture based
on ResNet. Furthermore, the new architecture is compared with ResNet and then
tested on ResNet-based networks. Massive experiments are conducted on CIFAR and
ImageNet, which witnesses the significant performance improvements provided by
the architecture.
