A New Type of Neurons for Machine Learning
  In machine learning, the use of an artificial neural network is the
mainstream approach. Such a network consists of layers of neurons. These
neurons are of the same type characterized by the two features: (1) an inner
product of an input vector and a matching weighting vector of trainable
parameters and (2) a nonlinear excitation function. Here we investigate the
possibility of replacing the inner product with a quadratic function of the
input vector, thereby upgrading the 1st order neuron to the 2nd order neuron,
empowering individual neurons, and facilitating the optimization of neural
networks. Also, numerical examples are provided to illustrate the feasibility
and merits of the 2nd order neurons. Finally, further topics are discussed.
