{'fraction_non_alphanumeric': 0.03067023232492061, 'fraction_numerical': 0.011281965569112486, 'mean_word_length': 5.037840565085772, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Large-Scale Unsupervised Deep Representation Learning for Brain Structure

Machine Learning (ML) is increasingly being used for computer aided diagnosis of brain related disorders based on structural magnetic resonance imaging (MRI) data. Most of such work employs biologically and medically meaningful hand-crafted features calculated from different regions of the brain. The construction of such highly specialized features requires a considerable amount of time, manual oversight and careful quality control to ensure the absence of errors in the computational process. Recent advances in Deep Representation Learning have shown great promise in extracting highly non-linear and information-rich features from data. In this paper, we present a novel large-scale deep unsupervised approach to learn generic feature representations of structural brain MRI scans, which requires no specialized domain knowledge or manual intervention. Our method produces low-dimensional representations of brain structure, which can be used to reconstruct brain images with very low error and exhibit performance comparable to FreeSurfer features on various classification tasks.


Introduction
Structural brain magnetic resonance imaging (sBMRI) helps medical practitioners make effective diagnoses of disorders by allowing the visualization of characteristics of their patients' brains and the detection of abnormalities. The curation of such data, where the diagnosis has already been made, provides opportunities to use Machine Learning (ML) methods to learn models from these examples and assist medical practitioners in making future diagnoses more efficiently.
Developing ML models for classifying a subject's brain as normal or having a certain disorder requires an appropriate feature representation of the brain. Software such as FreeSurfer 1 have been traditionally used to extract summary statistics from macro-scale brain regions that are then used to train classification models [7,12,13]. These features typically consist of surface area, volume and thickness of various regions of the brain. Although these specialized features are automatically computed by software, manual verification and careful quality control is required to ascertain that the process is error-free and to manually correct errors when they occur. Further, such software conform to a very strong prior over the brain, a very fixed view of anatomy, which is not correct, especially in abnormal cases where the structure of brains are deformed. Moreover, such a feature representation comprises a fixed set of targeted region properties, which fails to capture other potentially important information contained in the images.
We propose an unsupervised deep representation learning approach, based on convolutional autoencoders (CAEs) [8], to learn low-dimensional representations of brain structure from sBMRI scans. We present three CAE models that we employed in our work: CAE-staged, CAE-joint and CAE-3D. CAE-staged and CAE-joint treat 3D brain images as a series of 2D frames along the Z-axis. CAE-staged comprises two separate CAEs: one learns frame-level representations and the other learns a combined representation of the brain structure from the frame-level encodings. We merge the two autoencoders of CAE-staged together to create the CAE-joint network for end-to-end representation learning. CAE-3D operates on the 3D brain images using 3D operations to directly learn latent embeddings. Large amounts of data are required to train these networks due to their massive number of parameters, as is true with most deep learning models. However, most publicly available sBMRI datasets are relatively very small, on the spectrum of tens to a few thousands of images. Hence, we combine data from nine different sources to create a common dataset for unsupervised representation learning. The diversity induced from pooling data from multiple sources also makes the learned features more informative and robust. Features learned using our approach take considerably less time to construct and have performance comparable to those calculated using FreeSurfer (FS) on binary classification tasks based on identifying patients as healthy, having AD, having mild cognitive impairment (MCI), or having Autism Spectrum Disorder (ASD).
In the following sections, we present the data and preprocessing methods, conceptual understanding of CAEs, our CAE architectures, and their qualitative and quantitative analyses. To the best of our knowledge, our work is the first large-scale (spanning nine different datasets) unsupervised deep representation learning effort for structural brain images, paving the way for a plethora of possible future work, which is further motivated by our results.

Data
We train our deep CAE models on sBMRI scans, which are static 3D images of brains meant for capturing structural details, from nine different sources: -Alzheimer's Disease Neuroimaging Initiative (ADNI) 2 3 -for the study of mild cognitive impairment (MCI) and early Alzheimer's disease (AD). -Australian Imaging, Biomarker & Lifestyle Flagship Study of Ageing (AIBL) -for the study of AD [2].
-Open Access Series of Imaging Studies (OASIS) -for the study of AD.  4 -for the study of Autism Spectrum Disorder (ASD). -Brainomics/Localizer -for the study of inter-subject variability along different modalities of brain imaging [10,9]. -Human Connectome Project (HCP) -for the study of various functions [3].
-International Consortium for Brain Mapping (ICBM) 5 -for the development of a probabilistic reference system for the human brain.

Convolutional Autoencoders
An autoencoder [6] (AE) is a neural network composed of an encoder, which transforms data into its latent representation, and a decoder, which reconstructs data from its encoding. The model is trained to minimize the reconstruction loss. Convolutional Autoencoders (CAEs) [8] effectively capture frequently occurring local features through parameter sharing across the input by employing convolution layers in the encoder and deconvolution layers in the decoder. We use max-pooling layers in CAE encoders to gradually downsample data and unpooling layers in CAE decoders for progressive upsampling. We use the rectified linear activation function (described by y = max(x, 0)) at all the layers of our networks, and batch-normalization to speed up the training process.

CAE Architectures
CAE-staged. This model consists of two separate CAEs: CAE-staged-I and CAE-staged-II. Figure 1 describes their complete architectures as the left and the right halves of the entire graph. We first train CAE-staged-I to reconstruct any given frame so that it learns 50-dimensional frame-level representations. We then stack the 50-dimensional representations of all the 100 frames to create a (100 × 50) 2D representation of each brain. We train CAE-staged-II to learn a 50-dimensional latent representation of entire brains that can encode and reconstruct these (100 × 50) intermediate encodings.
CAE-joint. We create the CAE-joint model by merging CAE-staged-I and CAE-staged-II into a single network, as shown in Figure 1. We reshape 50dimensional frame-level encodings into (100 × 50) 2D brain representations in the encoder and the reconstructed (100 × 50) representations into reconstructed 50-dimensional frame-level encodings in the decoder. We use the learned weights from CAE-staged-I and CAE-staged-II as initial weights while training CAEjoint. CAE-joint also produces 50-dimensional encodings for entire brains. CAE-3D. This model is a single CAE which treats each brain as a 3D image. It uses the 3D versions of convolution, deconvolution, pooling and unpooling layers. Figure 2 describes the complete architecture of CAE-3D. We train it to directly learn a 50-dimensional representation of brain images that can encode and reconstruct 100 × 100 × 100 brain images wholly.

Qualitative Analysis
The original brain images have voxel values between 0 and 255. The voxel-wise reconstruction mean squared errors of CAE-staged-I, CAE-joint and CAE-3D on validation data are 4.43, 10.55 and 8.01, respectively. We visualize reconstructed brain images by plotting a few 2D frames from the 3D tensors. Figure 3 shows frames from original and reconstructed tensors generated by CAE-staged-I (first half of CAE-staged), CAE-joint and CAE-3D models. It can be seen that in all the three models, the learned representations can be used to generate high quality reconstructions of brain images.
To understand how our models extract features from 3D brain images, we infer and visualize the saliency maps of the nodes in the 50-dimensional brain representation layers of our models using the publicly available keras-vis 9 library. The saliency map of a node is calculated by taking the absolute value of the partial derivative of the node's value with respect to the input features. Hence, it indicates the magnitude of the effect of a voxel's perturbation on the node's activation. Figures 4,5 and 6 show the saliency maps of a few nodes in the embedding layers of CAE-staged, CAE-joint and CAE-3D, respectively. Each row in these figures corresponds to five frames of the saliency map of a single embedding node. The results show that all the three models capture features from highly-complex 3D substructures of brain images. The maps also reveal that each embedding node has different localized attention regions, which it accounts for in the information it captures. The saliency maps of the three models look dissimilar, indicating that they process brain images differently. The attention regions of CAE-3D look relatively more localized compared to those of CAE-staged and CAE-joint, indicating its stronger focus on local details.    [4] as the evaluation metric. We use 80% of the data for training and 20% for testing, split randomly. We summarize the results of our experiments on test data in Table 1. We see that the embeddings learned by our models show performance comparable to traditionally used FS features on all the aforementioned classification tasks, with CAE-3D performing slightly better than CAE-joint.
Timing Analysis. Feature construction using FreeSurfer takes 20−47 hours for each image 10 . In contrast, CAE-staged and CAE-joint take 0.55s, and CAE-3D takes 0.45s on average to generate the latent embedding of each brain image.

Related Work
Güçlü and Gerven [5] showed that unsupervised feature learning from functional brain MRI data improves human brain activity prediction in response to natural images. Brosch et al. [1] proposed a method for manifold learning of brain images in the ADNI dataset using Deep Belief Networks (DBNs) composed of convolutional Restricted Boltzmann Machines. However, their method is not end-to-end trainable and, hence, not scaleable. Plis et al. [11] presented the use of DBNs in a Constraint Satisfaction Problem framework to learn latent embeddings of gray matter images extracted from sBMRI scans on two datasets separately. Our work is different from theirs as we propose a large-scale deep unsupervised approach to learn latent representations of brain structure from sBMRI scans, spanning nine different datasets, without the need for first extracting gray matter images.

Conclusion
We presented an approach for large-scale deep unsupervised representation learning for sBMRI data incorporating models based on CAEs: CAE-staged, CAEjoint and CAE-3D. Features learned using our approach can reconstruct brain images with very low error and show performance comparable to FS features on classification tasks. Feature encoding using our method takes considerably less time compared to FS features, while employing no specialized domain knowledge.
Our models do not have a fixed view of brain anatomy and can be made increasingly general with diversified training data. The proposed models are adaptive, i.e., the learned features can be improved by training with more data.