Support vector machines and Radon's theorem
  A support vector machine (SVM) is an algorithm which finds a hyperplane that
optimally separates labeled data points in $\mathbb{R}^n$ into positive and
negative classes. The data points on the margin of this separating hyperplane
are called support vectors. We connect the possible configurations of support
vectors to Radon's theorem, which provides guarantees for when a set of points
can be divided into two classes (positive and negative) whose convex hulls
intersect. If the convex hulls of the positive and negative support vectors are
projected onto a separating hyperplane, the projections intersect in at least
one point if and only if the hyperplane is optimal. Further, with a particular
type of general position, we show that (a) the projected convex hulls of the
support vectors intersect in exactly one point, (b) the support vectors are
stable under perturbation, (c) there are at most $n+1$ support vectors, and (d)
every number of support vectors from 2 up to $n+1$ is possible. Finally, we
perform computer simulations studying the expected number of support vectors,
and their configurations, for randomly generated data. We observe that as the
distance between classes of points increases for this type of randomly
generated data, configurations with two support vectors become the most likely
configurations.
