Variance Based Samples Weighting for Supervised Deep Learning
  In the context of supervised learning of a function by a Neural Network (NN),
we claim and empirically justify that a NN yields better results when the
distribution of the data set focuses on regions where the function to learn is
steeper. We first traduce this assumption in a mathematically workable way
using Taylor expansion. Then, theoretical derivations allow to construct a
methodology that we call Variance Based Samples Weighting (VBSW). VBSW uses
local variance of the labels to weight the training points. This methodology is
general, scalable, cost effective, and significantly increases the performances
of a large class of NNs for various classification and regression tasks on
image, text and multivariate data. We highlight its benefits with experiments
involving NNs from shallow linear NN to Resnet or Bert.
