Revisiting Gaussian Neurons for Online Clustering with Unknown Number of Clusters
Despite the recent success of artificial neural networks, more biologically
plausible learning methods may be needed to resolve the weaknesses of
backpropagation trained models such as catastrophic forgetting and adversarial
attacks. A novel local learning rule is presented that performs online
clustering with a maximum limit of the number of cluster to be found rather
than a fixed cluster count. Instead of using orthogonal weight or output
activation constraints, activation sparsity is achieved by mutual repulsion of
lateral Gaussian neurons ensuring that multiple neuron centers cannot occupy
the same location in the input domain. An update method is also presented for
adjusting the widths of the Gaussian neurons in cases where the data samples
can be represented by means and variances. The algorithms were applied on the
MNIST and CIFAR-10 datasets to create filters capturing the input patterns of
pixel patches of various sizes. The experimental results demonstrate stability
in the learned parameters across a large number of training samples.