Robust learning of data anomalies with analytically-solvable entropic
  outlier sparsification
  Entropic Outlier Sparsification (EOS) is proposed as a robust computational
strategy for the detection of data anomalies in a broad class of learning
methods, including the unsupervised problems (like detection of non-Gaussian
outliers in mostly-Gaussian data) and in the supervised learning with
mislabeled data. EOS dwells on the derived analytic closed-form solution of the
(weighted) expected error minimization problem subject to the Shannon entropy
regularization. In contrast to common regularization strategies requiring
computational costs that scale polynomial with the data dimension, identified
closed-form solution is proven to impose additional iteration costs that depend
linearly on statistics size and are independent of data dimension. Obtained
analytic results also explain why the mixtures of spherically-symmetric
Gaussians - used heuristically in many popular data analysis algorithms -
represent an optimal choice for the non-parametric probability distributions
when working with squared Euclidean distances, combining expected error
minimality, maximal entropy/unbiasedness, and a linear cost scaling. The
performance of EOS is compared to a range of commonly-used tools on synthetic
problems and on partially-mislabeled supervised classification problems from
biomedicine.
