Numerical Computation of Partial Differential Equations by Hidden-Layer
  Concatenated Extreme Learning Machine
  The extreme learning machine (ELM) method can yield highly accurate solutions
to linear/nonlinear partial differential equations (PDEs), but requires the
last hidden layer of the neural network to be wide to achieve a high accuracy.
If the last hidden layer is narrow, the accuracy of the existing ELM method
will be poor, irrespective of the rest of the network configuration. In this
paper we present a modified ELM method, termed HLConcELM (hidden-layer
concatenated ELM), to overcome the above drawback of the conventional ELM
method. The HLConcELM method can produce highly accurate solutions to
linear/nonlinear PDEs when the last hidden layer of the network is narrow and
when it is wide. The new method is based on a type of modified feedforward
neural networks (FNN), termed HLConcFNN (hidden-layer concatenated FNN), which
incorporates a logical concatenation of the hidden layers in the network and
exposes all the hidden nodes to the output-layer nodes. HLConcFNNs have the
interesting property that, given a network architecture, when additional hidden
layers are appended to the network or when extra nodes are added to the
existing hidden layers the representation capacity of the HLConcFNN associated
with the new architecture is guaranteed to be not smaller than that of the
original network architecture. Here representation capacity refers to the set
of all functions that can be exactly represented by the neural network of a
given architecture. We present ample benchmark tests with linear/nonlinear PDEs
to demonstrate the computational accuracy and performance of the HLConcELM
method and the superiority of this method to the conventional ELM from previous
works.
