Use of Deterministic Transforms to Design Weight Matrices of a Neural
  Network
  Self size-estimating feedforward network (SSFN) is a feedforward multilayer
network. For the existing SSFN, a part of each weight matrix is trained using a
layer-wise convex optimization approach (a supervised training), while the
other part is chosen as a random matrix instance (an unsupervised training). In
this article, the use of deterministic transforms instead of random matrix
instances for the SSFN weight matrices is explored. The use of deterministic
transforms provides a reduction in computational complexity. The use of several
deterministic transforms is investigated, such as discrete cosine transform,
Hadamard transform, Hartley transform, and wavelet transforms. The choice of a
deterministic transform among a set of transforms is made in an unsupervised
manner. To this end, two methods based on features' statistical parameters are
developed. The proposed methods help to design a neural net where deterministic
transforms can vary across its layers' weight matrices. The effectiveness of
the proposed approach vis-a-vis the SSFN is illustrated for object
classification tasks using several benchmark datasets.
