PairNets: Novel Fast Shallow Artificial Neural Networks on Partitioned
  Subspaces
  Traditionally, an artificial neural network (ANN) is trained slowly by a
gradient descent algorithm such as the backpropagation algorithm since a large
number of hyperparameters of the ANN need to be fine-tuned with many training
epochs. To highly speed up training, we created a novel shallow 4-layer ANN
called "Pairwise Neural Network" ("PairNet") with high-speed hyperparameter
optimization. In addition, a value of each input is partitioned into multiple
intervals, and then an n-dimensional space is partitioned into M n-dimensional
subspaces. M local PairNets are built in M partitioned local n-dimensional
subspaces. A local PairNet is trained very quickly with only one epoch since
its hyperparameters are directly optimized one-time via simply solving a system
of linear equations by using the multivariate least squares fitting method.
Simulation results for three regression problems indicated that the PairNet
achieved much higher speeds and lower average testing mean squared errors
(MSEs) for the three cases, and lower average training MSEs for two cases than
the traditional ANNs. A significant future work is to develop better and faster
optimization algorithms based on intelligent methods and parallel computing
methods to optimize both partitioned subspaces and hyperparameters to build the
fast and effective PairNets for applications in big data mining and real-time
machine learning.
