{'fraction_non_alphanumeric': 0.031857031857031856, 'fraction_numerical': 0.00777000777000777, 'mean_word_length': 5.28515625, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Tissue segmentation with deep 3D networks and spatial priors

Conventional automated segmentation of the human head distinguishes different tissues based on image intensities in an MRI volume and prior tissue probability maps (TPM). This works well for normal head anatomies, but fails in the presence of unexpected lesions. Deep convolutional neural networks leverage instead volumetric spatial patterns and can be trained to segment lesions, but have thus far not integrated prior probabilities. Here we add to a three-dimensional convolutional network spatial priors with a TPM, morphological priors with conditional random fields, and context with a wider field-of-view at lower resolution. The new architecture, which we call MultiPrior, was designed to be a fully-trainable, three-dimensional convolutional network. Thus, the resulting architecture represents a neural network with learnable spatial memories. When trained on a set of stroke patients and healthy subjects, MultiPrior outperforms the state-of-the-art segmentation tools such as DeepMedic and SPM segmentation. The approach is further demonstrated on patients with disorders of consciousness, where we find that cognitive state correlates positively with gray-matter volumes and negatively with the extent of ventricles. We make the code and trained networks freely available to support future clinical research projects. INTRODUCTION Clinical and basic research require segmentation of magnetic resonance images (MRI) of human heads, including abnormal anatomies such as tumors or lesions. In the case of brain tumors, for example, it is helpful to measure the tumor volume across successive scans to monitor tumor growth or the response to treatment ​(1,2)​. In the case of a stroke, lesion studies can reveal important insights on brain function ​(3,4)​. In large populations this is only possible with automated segmentation ​(5–8)​. Finally, neuromodulation techniques such as transcranial electric stimulation rely on accurate head segmentations to numerically calculate the optimal stimulation montages ​(9,10)​. Such a segmentation needs to capture not only the brain and lesion but also the cerebrospinal fluid, ventricles, air cavities, skull, etc ​(11,12)​. A number of free tools have been developed to automate the task of segmenting the brain. This includes algorithms that are part of neuroimaging software packages such as SPM ​(13) and FSL ​(14)​. These algorithms traditionally distinguish different tissues based on the brightness of voxels, e.g., T1-weighted MRI: white matter is bright, gray matter is gray, and surrounding CSF and skull are black. Segmentation also relies on prior information on the type of tissue expected at different location of the head, e.g., with very high probability the surface of the head is skin. These probabilities are derived from large dataset of normal human anatomies and are stored as tissue probability maps (TPM) ​(13)​. Traditional algorithms that are based on intensity and location work well for subjects with normal anatomy, but often fail in the presence of lesions. For example, in chronic stroke patients, areas that typically contain brain (bright) are filled with cerebrospinal fluid (dark in T1 images). This leads to ambiguities that confused the traditional algorithms and can result in errors that extend beyond the lesions. As a result, MRIs from stroke patients require manual correction of the automated segmentations, despite efforts to improve on these traditional tools ​(6–8)​. A breakthrough has come from deep convolutional neural networks that can be trained to identify different tissue types based on their appearance, i.e. not just intensity but complex three-dimensional intensity features. For example, DeepMedic or QuickNAT are deep 3D convolutional networks that achieve good segmentation of stroke lesions or brain tumors ​(15,16)​. To-date these new network approaches have not leveraged spatial or morphological prior information, which have greatly benefited traditional intensity-based methods ​(17)​. Here we propose to leverage spatial prior information by including a TPM as additional input to a convolutional 3D network. We also leverage morphological priors by adding 3D Conditional Random Fields (CRF) as additional processing of the output of the network similar to ​(15)​. To do this we extend existing 2D implementations of dense CRF to 3D. Our CRF uses a morphological model as penalty term instead of the commonly used Potts-model ​(18)​. Following the approach of DeepMedic ​(15,19) we also allow the network to learn context information by providing a low-resolution, but wider field-of-view at the input. We call this new deep, three-dimensional network, MultiPrior. This new convolutional network can be trained to segment the MRI volume based on 3D appearance, but it also takes voxel location with its prior tissue probability into account. This network is no longer shift-invariant and can thus learn to interpret appearance differently at different locations of the head. The CRF implements morphological constraints at the output. For instance, it does not allow segmentations where the brain touches the skin, as we know a priori that there must be at least skull between brain and skin ​(17)​ . The resulting architecture is in principle fully trainable, including the prior probabilities. Thus, the MultiPrior architecture represents a convolutional network with learnable spatial memories. These memories have a simple interpretation, and can be manually instantiated based on prior spatial knowledge. Here we train the networks on available manual segmentations including stroke patients and normal subjects. Based on this ground truth we objectively judge the benefits of adding prior information and spatial context. The largest gains are obtained by adding context or spatial priors. When combining the three techniques, the resulting MultiPrior network outperforms state-of-the-art methods such as DeepMedic and SPM segmentation on these data. The utility of the technique is demonstrated on a separate data set of patients with disorders of consciousness. We find volumetric differences between vegetative and minimally conscious patients, consistent with prior literature ​(20)​. We make the MultiPrior tool freely available (21)​in the hope that this will spawn further development and that it will be used in clinical research project.