Evenly Cascaded Convolutional Networks
  We introduce Evenly Cascaded convolutional Network (ECN), a neural network
taking inspiration from the cascade algorithm of wavelet analysis. ECN employs
two feature streams - a low-level and high-level steam. At each layer these
streams interact, such that low-level features are modulated using advanced
perspectives from the high-level stream. ECN is evenly structured through
resizing feature map dimensions by a consistent ratio, which removes the burden
of ad-hoc specification of feature map dimensions. ECN produces easily
interpretable features maps, a result whose intuition can be understood in the
context of scale-space theory. We demonstrate that ECN's design facilitates the
training process through providing easily trainable shortcuts. We report new
state-of-the-art results for small networks, without the need for additional
treatment such as pruning or compression - a consequence of ECN's simple
structure and direct training. A 6-layered ECN design with under 500k
parameters achieves 95.24% and 78.99% accuracy on CIFAR-10 and CIFAR-100
datasets, respectively, outperforming the current state-of-the-art on small
parameter networks, and a 3 million parameter ECN produces results competitive
to the state-of-the-art.
