High-dimensional separability for one- and few-shot learning
  This work is driven by a practical question: corrections of Artificial
Intelligence (AI) errors. These corrections should be quick and non-iterative.
To solve this problem without modification of a legacy AI system, we propose
special `external' devices, correctors. Elementary correctors consist of two
parts, a classifier that separates the situations with high risk of error from
the situations in which the legacy AI system works well and a new decision for
situations with potential errors. Input signals for the correctors can be the
inputs of the legacy AI system, its internal signals, and outputs. If the
intrinsic dimensionality of data is high enough then the classifiers for
correction of small number of errors can be very simple. According to the
blessing of dimensionality effects, even simple and robust Fisher's
discriminants can be used for one-shot learning of AI correctors. Stochastic
separation theorems provide the mathematical basis for this one-short learning.
However, as the number of correctors needed grows, the cluster structure of
data becomes important and a new family of stochastic separation theorems is
required. We refuse the classical hypothesis of the regularity of the data
distribution and assume that the data can have a fine-grained structure with
many clusters and peaks in the probability density. New stochastic separation
theorems for data with fine-grained structure are formulated and proved. The
multi-correctors for granular data are proposed. The advantages of the
multi-corrector technology were demonstrated by examples of correcting errors
and learning new classes of objects by a deep convolutional neural network on
the CIFAR-10 dataset. The key problems of the non-classical high-dimensional
data analysis are reviewed together with the basic preprocessing steps
including supervised, semi-supervised and domain adaptation Principal Component
Analysis.
