A Modified Sigma-Pi-Sigma Neural Network with Adaptive Choice of
  Multinomials
  Sigma-Pi-Sigma neural networks (SPSNNs) as a kind of high-order neural
networks can provide more powerful mapping capability than the traditional
feedforward neural networks (Sigma-Sigma neural networks). In the existing
literature, in order to reduce the number of the Pi nodes in the Pi layer, a
special multinomial P_s is used in SPSNNs. Each monomial in P_s is linear with
respect to each particular variable sigma_i when the other variables are taken
as constants. Therefore, the monomials like sigma_i^n or sigma_i^n sigma_j with
n>1 are not included. This choice may be somehow intuitive, but is not
necessarily the best. We propose in this paper a modified Sigma-Pi-Sigma neural
network (MSPSNN) with an adaptive approach to find a better multinomial for a
given problem. To elaborate, we start from a complete multinomial with a given
order. Then we employ a regularization technique in the learning process for
the given problem to reduce the number of monomials used in the multinomial,
and end up with a new SPSNN involving the same number of monomials (= the
number of nodes in the Pi-layer) as in P_s. Numerical experiments on some
benchmark problems show that our MSPSNN behaves better than the traditional
SPSNN with P_s.
