SplineNets: Continuous Neural Decision Graphs
  We present SplineNets, a practical and novel approach for using conditioning
in convolutional neural networks (CNNs). SplineNets are continuous
generalizations of neural decision graphs, and they can dramatically reduce
runtime complexity and computation costs of CNNs, while maintaining or even
increasing accuracy. Functions of SplineNets are both dynamic (i.e.,
conditioned on the input) and hierarchical (i.e., conditioned on the
computational path). SplineNets employ a unified loss function with a desired
level of smoothness over both the network and decision parameters, while
allowing for sparse activation of a subset of nodes for individual samples. In
particular, we embed infinitely many function weights (e.g. filters) on smooth,
low dimensional manifolds parameterized by compact B-splines, which are indexed
by a position parameter. Instead of sampling from a categorical distribution to
pick a branch, samples choose a continuous position to pick a function weight.
We further show that by maximizing the mutual information between spline
positions and class labels, the network can be optimally utilized and
specialized for classification tasks. Experiments show that our approach can
significantly increase the accuracy of ResNets with negligible cost in speed,
matching the precision of a 110 level ResNet with a 32 level SplineNet.
