Being Friends Instead of Adversaries: Deep Networks Learn from Data
  Simplified by Other Networks
  Amongst a variety of approaches aimed at making the learning procedure of
neural networks more effective, the scientific community developed strategies
to order the examples according to their estimated complexity, to distil
knowledge from larger networks, or to exploit the principles behind adversarial
machine learning. A different idea has been recently proposed, named Friendly
Training, which consists in altering the input data by adding an automatically
estimated perturbation, with the goal of facilitating the learning process of a
neural classifier. The transformation progressively fades-out as long as
training proceeds, until it completely vanishes. In this work we revisit and
extend this idea, introducing a radically different and novel approach inspired
by the effectiveness of neural generators in the context of Adversarial Machine
Learning. We propose an auxiliary multi-layer network that is responsible of
altering the input data to make them easier to be handled by the classifier at
the current stage of the training procedure. The auxiliary network is trained
jointly with the neural classifier, thus intrinsically increasing the 'depth'
of the classifier, and it is expected to spot general regularities in the data
alteration process. The effect of the auxiliary network is progressively
reduced up to the end of training, when it is fully dropped and the classifier
is deployed for applications. We refer to this approach as Neural Friendly
Training. An extended experimental procedure involving several datasets and
different neural architectures shows that Neural Friendly Training overcomes
the originally proposed Friendly Training technique, improving the
generalization of the classifier, especially in the case of noisy data.
