Diffusion Mechanism in Residual Neural Network: Theory and Applications
  Diffusion, a fundamental internal mechanism emerging in many physical
processes, describes the interaction among different objects. In many learning
tasks with limited training samples, the diffusion connects the labeled and
unlabeled data points and is a critical component for achieving high
classification accuracy. Many existing deep learning approaches directly impose
the fusion loss when training neural networks. In this work, inspired by the
convection-diffusion ordinary differential equations (ODEs), we propose a novel
diffusion residual network (Diff-ResNet), internally introduces diffusion into
the architectures of neural networks. Under the structured data assumption, it
is proved that the proposed diffusion block can increase the distance-diameter
ratio that improves the separability of inter-class points and reduces the
distance among local intra-class points. Moreover, this property can be easily
adopted by the residual networks for constructing the separable hyperplanes.
Extensive experiments of synthetic binary classification, semi-supervised graph
node classification and few-shot image classification in various datasets
validate the effectiveness of the proposed method.
