Geometry of Deep Convolutional Networks
  We give a formal procedure for computing preimages of convolutional network
outputs using the dual basis defined from the set of hyperplanes associated
with the layers of the network. We point out the special symmetry associated
with arrangements of hyperplanes of convolutional networks that take the form
of regular multidimensional polyhedral cones. We discuss the efficiency of
large number of layers of nested cones that result from incremental small size
convolutions in order to give a good compromise between efficient contraction
of data to low dimensions and shaping of preimage manifolds. We demonstrate how
a specific network flattens a non linear input manifold to an affine output
manifold and discuss its relevance to understanding classification properties
of deep networks.
