Self-Reorganizing and Rejuvenating CNNs for Increasing Model Capacity
  Utilization
  In this paper, we propose self-reorganizing and rejuvenating convolutional
neural networks; a biologically inspired method for improving the computational
resource utilization of neural networks. The proposed method utilizes the
channel activations of a convolution layer in order to reorganize that layers
parameters. The reorganized parameters are clustered to avoid parameter
redundancies. As such, redundant neurons with similar activations are merged
leaving room for the remaining parameters to rejuvenate. The rejuvenated
parameters learn different features to supplement those learned by the
reorganized surviving parameters. As a result, the network capacity utilization
increases improving the baseline network performance without any changes to the
network structure. The proposed method can be applied to various network
architectures during the training stage, or applied to a pre-trained model
improving its performance. Experimental results showed that the proposed method
is model-agnostic and can be applied to any backbone architecture increasing
its performance due to the elevated utilization of the network capacity.
