Learning Data Manifolds with a Cutting Plane Method
  We consider the problem of classifying data manifolds where each manifold
represents invariances that are parameterized by continuous degrees of freedom.
Conventional data augmentation methods rely upon sampling large numbers of
training examples from these manifolds; instead, we propose an iterative
algorithm called M_{CP} based upon a cutting-plane approach that efficiently
solves a quadratic semi-infinite programming problem to find the maximum margin
solution. We provide a proof of convergence as well as a polynomial bound on
the number of iterations required for a desired tolerance in the objective
function. The efficiency and performance of M_{CP} are demonstrated in
high-dimensional simulations and on image manifolds generated from the ImageNet
dataset. Our results indicate that M_{CP} is able to rapidly learn good
classifiers and shows superior generalization performance compared with
conventional maximum margin methods using data augmentation methods.
