Stochastic Separation Theorems
  The problem of non-iterative one-shot and non-destructive correction of
unavoidable mistakes arises in all Artificial Intelligence applications in the
real world. Its solution requires robust separation of samples with errors from
samples where the system works properly. We demonstrate that in (moderately)
high dimension this separation could be achieved with probability close to one
by linear discriminants. Surprisingly, separation of a new image from a very
large set of known images is almost always possible even in moderately high
dimensions by linear functionals, and coefficients of these functionals can be
found explicitly. Based on fundamental properties of measure concentration, we
show that for $M<a\exp(b{n})$ random $M$-element sets in $\mathbb{R}^n$ are
linearly separable with probability $p$, $p>1-\vartheta$, where $1>\vartheta>0$
is a given small constant. Exact values of $a,b>0$ depend on the probability
distribution that determines how the random $M$-element sets are drawn, and on
the constant $\vartheta$. These {\em stochastic separation theorems} provide a
new instrument for the development, analysis, and assessment of machine
learning methods and algorithms in high dimension. Theoretical statements are
illustrated with numerical examples.
