A Heterogeneous Graph Based Framework for Multimodal Neuroimaging Fusion
  Learning
  Here, we present a Heterogeneous Graph neural network for Multimodal
neuroimaging fusion learning (HGM). Traditional GNN-based models usually assume
the brain network is a homogeneous graph with single type of nodes and edges.
However, vast literatures have shown the heterogeneity of the human brain
especially between the two hemispheres. Homogeneous brain network is
insufficient to model the complicated brain state. Therefore, in this work we
firstly model the brain network as heterogeneous graph with multi-type nodes
(i.e., left and right hemispheric nodes) and multi-type edges (i.e., intra- and
inter-hemispheric edges). Besides, we also propose a self-supervised
pre-training strategy based on heterogeneou brain network to address the
overfitting problem due to the complex model and small sample size. Our results
on two datasets show the superiority of proposed model over other multimodal
methods for disease prediction task. Besides, ablation experiments show that
our model with pre-training strategy can alleviate the problem of limited
training sample size.
