A Deep-Generative Hybrid Model to Integrate Multimodal and Dynamic
  Connectivity for Predicting Spectrum-Level Deficits in Autism
  We propose an integrated deep-generative framework, that jointly models
complementary information from resting-state functional MRI (rs-fMRI)
connectivity and diffusion tensor imaging (DTI) tractography to extract
predictive biomarkers of a disease. The generative part of our framework is a
structurally-regularized Dynamic Dictionary Learning (sr-DDL) model that
decomposes the dynamic rs-fMRI correlation matrices into a collection of shared
basis networks and time varying patient-specific loadings. This matrix
factorization is guided by the DTI tractography matrices to learn anatomically
informed connectivity profiles. The deep part of our framework is an LSTM-ANN
block, which models the temporal evolution of the patient sr-DDL loadings to
predict multidimensional clinical severity. Our coupled optimization procedure
collectively estimates the basis networks, the patient-specific dynamic
loadings, and the neural network weights. We validate our framework on a
multi-score prediction task in 57 patients diagnosed with Autism Spectrum
Disorder (ASD). Our hybrid model outperforms state-of-the-art baselines in a
five-fold cross validated setting and extracts interpretable multimodal neural
signatures of brain dysfunction in ASD.
