Deep Stochastic Configuration Networks with Universal Approximation
  Property
  This paper develops a randomized approach for incrementally building deep
neural networks, where a supervisory mechanism is proposed to constrain the
random assignment of the weights and biases, and all the hidden layers have
direct links to the output layer. A fundamental result on the universal
approximation property is established for such a class of randomized leaner
models, namely deep stochastic configuration networks (DeepSCNs). A learning
algorithm is presented to implement DeepSCNs with either specific architecture
or self-organization. The read-out weights attached with all direct links from
each hidden layer to the output layer are evaluated by the least squares
method. Given a set of training examples, DeepSCNs can speedily produce a
learning representation, that is, a collection of random basis functions with
the cascaded inputs together with the read-out weights. An empirical study on a
function approximation is carried out to demonstrate some properties of the
proposed deep learner model.
