Deep Neural Nets with Interpolating Function as Output Activation
  We replace the output layer of deep neural nets, typically the softmax
function, by a novel interpolating function. And we propose end-to-end training
and testing algorithms for this new architecture. Compared to classical neural
nets with softmax function as output activation, the surrogate with
interpolating function as output activation combines advantages of both deep
and manifold learning. The new framework demonstrates the following major
advantages: First, it is better applicable to the case with insufficient
training data. Second, it significantly improves the generalization accuracy on
a wide variety of networks. The algorithm is implemented in PyTorch, and code
will be made publicly available.
