A Constructive Approach for Data-Driven Randomized Learning of
  Feedforward Neural Networks
  Feedforward neural networks with random hidden nodes suffer from a problem
with the generation of random weights and biases as these are difficult to set
optimally to obtain a good projection space. Typically, random parameters are
drawn from an interval which is fixed before or adapted during the learning
process. Due to the different functions of the weights and biases, selecting
them both from the same interval is not a good idea. Recently more
sophisticated methods of random parameters generation have been developed, such
as the data-driven method proposed in \cite{Anon19}, where the sigmoids are
placed in randomly selected regions of the input space and then their slopes
are adjusted to the local fluctuations of the target function. In this work, we
propose an extended version of this method, which constructs iteratively the
network architecture. This method successively generates new hidden nodes and
accepts them if the training error decreases significantly. The threshold of
acceptance is adapted to the current training stage. At the beginning of the
training process only those nodes which lead to the largest error reduction are
accepted. Then, the threshold is reduced by half to accept those nodes which
model the target function details more accurately. This leads to faster
convergence and more compact network architecture, as it includes only
"significant" neurons. Several application examples are given which confirm
this thesis.
