Friendly Training: Neural Networks Can Adapt Data To Make Learning
  Easier
  In the last decade, motivated by the success of Deep Learning, the scientific
community proposed several approaches to make the learning procedure of Neural
Networks more effective. When focussing on the way in which the training data
are provided to the learning machine, we can distinguish between the classic
random selection of stochastic gradient-based optimization and more involved
techniques that devise curricula to organize data, and progressively increase
the complexity of the training set. In this paper, we propose a novel training
procedure named Friendly Training that, differently from the aforementioned
approaches, involves altering the training examples in order to help the model
to better fulfil its learning criterion. The model is allowed to simplify those
examples that are too hard to be classified at a certain stage of the training
procedure. The data transformation is controlled by a developmental plan that
progressively reduces its impact during training, until it completely vanishes.
In a sense, this is the opposite of what is commonly done in order to increase
robustness against adversarial examples, i.e., Adversarial Training.
Experiments on multiple datasets are provided, showing that Friendly Training
yields improvements with respect to informed data sub-selection routines and
random selection, especially in deep convolutional architectures. Results
suggest that adapting the input data is a feasible way to stabilize learning
and improve the generalization skills of the network.
