Improving learnability of neural networks: adding supplementary axes to
  disentangle data representation
  Over-parameterized deep neural networks have proven to be able to learn an
arbitrary dataset with 100$\%$ training accuracy. Because of a risk of
overfitting and computational cost issues, we cannot afford to increase the
number of network nodes if we want achieve better training results for medical
images. Previous deep learning research shows that the training ability of a
neural network improves dramatically (for the same epoch of training) when a
few nodes with supplementary information are added to the network. These few
informative nodes allow the network to learn features that are otherwise
difficult to learn by generating a disentangled data representation. This paper
analyzes how concatenation of additional information as supplementary axes
affects the training of the neural networks. This analysis was conducted for a
simple multilayer perceptron (MLP) classification model with a rectified linear
unit (ReLU) on two-dimensional training data. We compared the networks with and
without concatenation of supplementary information to support our analysis. The
model with concatenation showed more robust and accurate training results
compared to the model without concatenation. We also confirmed that our
findings are valid for deeper convolutional neural networks (CNN) using
ultrasound images and for a conditional generative adversarial network (cGAN)
using the MNIST data.
