Competitive Learning Enriches Learning Representation and Accelerates
  the Fine-tuning of CNNs
  In this study, we propose the integration of competitive learning into
convolutional neural networks (CNNs) to improve the representation learning and
efficiency of fine-tuning. Conventional CNNs use back propagation learning, and
it enables powerful representation learning by a discrimination task. However,
it requires huge amount of labeled data, and acquisition of labeled data is
much harder than that of unlabeled data. Thus, efficient use of unlabeled data
is getting crucial for DNNs. To address the problem, we introduce unsupervised
competitive learning into the convolutional layer, and utilize unlabeled data
for effective representation learning. The results of validation experiments
using a toy model demonstrated that strong representation learning effectively
extracted bases of images into convolutional filters using unlabeled data, and
accelerated the speed of the fine-tuning of subsequent supervised back
propagation learning. The leverage was more apparent when the number of filters
was sufficiently large, and, in such a case, the error rate steeply decreased
in the initial phase of fine-tuning. Thus, the proposed method enlarged the
number of filters in CNNs, and enabled a more detailed and generalized
representation. It could provide a possibility of not only deep but broad
neural networks.
