Convexifying Sparse Interpolation with Infinitely Wide Neural Networks:
  An Atomic Norm Approach
  This work examines the problem of exact data interpolation via sparse (neuron
count), infinitely wide, single hidden layer neural networks with leaky
rectified linear unit activations. Using the atomic norm framework of
[Chandrasekaran et al., 2012], we derive simple characterizations of the convex
hulls of the corresponding atomic sets for this problem under several different
constraints on the weights and biases of the network, thus obtaining equivalent
convex formulations for these problems. A modest extension of our proposed
framework to a binary classification problem is also presented. We explore the
efficacy of the resulting formulations experimentally, and compare with
networks trained via gradient descent.
