Ensemble Deep Learning on Large, Mixed-Site fMRI Datasets in Autism and
  Other Tasks
  Deep learning models for MRI classification face two recurring problems: they
are typically limited by low sample size, and are abstracted by their own
complexity (the "black box problem"). In this paper, we train a convolutional
neural network (CNN) with the largest multi-source, functional MRI (fMRI)
connectomic dataset ever compiled, consisting of 43,858 datapoints. We apply
this model to a cross-sectional comparison of autism (ASD) vs typically
developing (TD) controls that has proved difficult to characterise with
inferential statistics. To contextualise these findings, we additionally
perform classifications of gender and task vs rest. Employing class-balancing
to build a training set, we trained 3$\times$300 modified CNNs in an ensemble
model to classify fMRI connectivity matrices with overall AUROCs of 0.6774,
0.7680, and 0.9222 for ASD vs TD, gender, and task vs rest, respectively.
Additionally, we aim to address the black box problem in this context using two
visualization methods. First, class activation maps show which functional
connections of the brain our models focus on when performing classification.
Second, by analyzing maximal activations of the hidden layers, we were also
able to explore how the model organizes a large and mixed-centre dataset,
finding that it dedicates specific areas of its hidden layers to processing
different covariates of data (depending on the independent variable analyzed),
and other areas to mix data from different sources. Our study finds that deep
learning models that distinguish ASD from TD controls focus broadly on temporal
and cerebellar connections, with a particularly high focus on the right caudate
nucleus and paracentral sulcus.
