{'fraction_non_alphanumeric': 0.03231626768051036, 'fraction_numerical': 0.01298450236814125, 'mean_word_length': 4.82545045045045, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 1, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 1, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Auto-context Convolutional Neural Network for Geometry-Independent Brain Extraction in Magnetic Resonance Imaging

Brain extraction or whole brain segmentation is an important first step in many of the neuroimage analysis pipelines. The accuracy and robustness of brain extraction, therefore, is crucial for the accuracy of the entire brain analysis process. With the aim of designing a learning-based, geometry-independent and registration-free brain extraction tool in this study, we present a technique based on an auto-context convolutional neural network (CNN), in which intrinsic local and global image features are learned through 2D patches of different window sizes. In this architecture three parallel 2D convolutional pathways for three different directions (axial, coronal, and sagittal) implicitly learn 3D image information without the need for computationally expensive 3D convolutions. Posterior probability maps generated by the network are used iteratively as context information along with the original image patches to learn the local shape and connectedness of the brain, to extract it from non-brain tissue. The brain extraction results we have obtained from our algorithm are superior to the recently reported results in the literature on two publicly available benchmark datasets, namely LPBA40 and OASIS, in which we obtained Dice overlap coefficients of 97.42% and 95.40%, respectively. Furthermore, we evaluated the performance of our algorithm in the challenging problem of extracting arbitrarily-oriented fetal brains in reconstructed fetal brain magnetic resonance imaging (MRI) datasets. In this application our algorithm performed much better than the other methods (Dice coefficient: 95.98%), where the other methods performed poorly due to the non-standard orientation and geometry of the fetal brain in MRI. Our CNN-based method can provide accurate, geometry-independent brain extraction in challenging applications.

Brain extraction or whole brain segmentation is an important first step in many of the neuroimage analysis pipelines. The accuracy and robustness of brain extraction, therefore, is crucial for the accuracy of the entire brain analysis process. State-of-the-art brain extraction techniques rely heavily on the accuracy of alignment or registration between brain atlases and query brain anatomy, and/or make strong assumptions about the image geometry; therefore have limited success when these assumptions do not hold or image registration fails. With the aim of designing a learning-based, geometry-independent and registration-free brain extraction tool in this study, we present a technique based on an auto-context convolutional neural network (CNN), in which intrinsic local and global image features are learned through 2D patches of different window sizes. In this architecture three parallel 2D convolutional pathways for three different directions (axial, coronal, and sagittal) implicitly learn 3D image information without the need for computationally expensive 3D convolutions. Posterior probability maps generated by the network are used iteratively as context information along with the original image patches to learn the local shape and connectedness of the brain, to extract it from non-brain tissue.
The brain extraction results we have obtained from our algorithm are superior to the recently reported results in the literature on two publicly available benchmark datasets, namely LPBA40 and OASIS, in which we obtained Dice overlap coefficients of 97.42% and 95.40%, respectively. Furthermore, we evaluated the performance of our algorithm in the challenging problem of extracting arbitrarily-oriented fetal brains in reconstructed fetal brain magnetic resonance imaging (MRI) datasets. In this application our algorithm performed much better than the other methods (Dice coefficient: 95.98%), where the other methods performed poorly due to the non-standard orientation and geometry of the fetal brain in MRI. Our CNN-based method can provide accurate, geometry-independent brain extraction in challenging applications. This in-turn may reduce the problems associated with image registration in segmentation tasks.

I. INTRODUCTION
W HOLE brain segmentation, or brain extraction, is one of the first fundamental steps in the analysis of magnetic resonance images (MRI) in advanced neuroimaging Manuscript received March 6, 2017. Corresponding author: S.S.M.Salehi (email: ssalehi@ece.neu.edu). Relevant code is disseminated at: https://github.com/SadeghMSalehi/AutoContextCNN applications such as brain tissue segmentation and volumetric analysis [1], longitudinal and group analysis [2], cortical and sub-cortical surface analysis and thickness measurement [3], [4], and surgical planning. Manual brain extraction is time consuming especially in large-scale studies. Automated brain extraction is necessary but its performance and accuracy is critical as the output of this step can directly affect the performance of all next steps.
Recently neural networks and deep learning have attracted enormous attention in medical image processing. Brebisson et.al. [5] proposed the SegNet, a convolutional neural network system to segment different parts of the brain. CNN-based methods have also been recently used successfully in tumor segmentation [6], [7], [8], brain lesion segmentation [9], [10], and infant brain image segmentation [11]. In what follows we review the state-of-the-art in whole brain segmentation and the related work that motivated this study. We then introduce a CNN-based method that generates accurate brain extraction.
II. RELATED WORK Many algorithms have been developed and continuously improved over the past decade for whole brain segmentation, which has been a necessary component of large-scale neuroscience and neuroimage analysis studies. As the usage of these algorithms dramatically grew, the demand for higher accuracy and reliability also increased. Consequently, while fully-automated, accurate brain extraction has already been investigated extensively, it is still an active area of research. Of particular interest is a recent deep learning based algorithm [12] that has shown to outperform most of the popular routinely-used brain extraction tools.
The state-of-the-art brain extraction methods and tools use evolved combinations of image registration, atlases, intensity and edge feature information, and level sets/graph cuts to generate brain masks in MRI images. The majority of these algorithms rely heavily on the alignment of the query images to atlases or make strong assumptions about the geometry, orientation, and image features. Yet the outcome of most of these tools is often inaccurate and involves non-brain structures or cuts parts of the brain. Therefore most of these tools offer options and multiple parameters to set and try, that ultimately make brain extraction a semi-automatic or supervised task rather than fully automatic. arXiv:1703.02083v1 [cs.CV] 6 Mar 2017 Among brain extraction methods four algorithms that are distributed with the widely-used neuroimage analysis software packages, have been evolved and are routinely used. These are the Brain Extraction Tool (BET) from FSL [13], [14], 3dSkullStrip from the AFNI toolkit [15], the Hybrid Watershed Algorithm (HWA) from FreeSurfer [16], and Robust Learning-Based Brain Extraction (ROBEX) [17]. BET expands a deformable spherical surface mesh model initialized at the center-of-gravity of the image based on local intensity values and surface smoothness. 3dSkullStrip, which is a modified version of BET, uses points outside of the expanding mesh to guide the borders of the mesh. HWA uses edge detection for watershed segmentation along with an atlas-based deformable surface model. ROBEX fits a triangular mesh, constrained by a shape model, to the probabilistic output of a brain boundary classifier based on random forests. Because the shape model alone cannot perfectly accommodate unseen cases, Robex also uses a small free-form deformation which is optimized via graph cuts.
The current methods are prone to significant errors when certain geometric assumptions do not hold, features are not precisely identified, or image registration, which is often not guaranteed to converge to an exact solution, fails. The problems associated with registration-based segmentation, and the recent promising results in neural network based image segmentation motivates further development and use of learning-based, geometry-independent, and registration-free brain image segmentation.
Recently, Kleesiek et. al. [12] proposed a deep learning based algorithm for brain extraction, which will be referred to as PCNN in this paper. PCNN uses seven 3D convolutional layers for voxelwise image segmentation. Cubes of size 53×53×53 around the grayscale target voxel are used as inputs to the network. In the extensive evaluation and comparison reported in [12], PCNN outperformed state-of-the-art brain extraction algorithms in publicly available benchmark datasets.
In this study we introduce a CNN-based method with three main contributions to further improve brain extraction accuracy. First, instead of using 3D convolutional layers with one window size, we use 2D patches of three different sizes as recently proposed by Moeskops et al. [18]; second, to account for 3D structure, and efficiently learn from 3D information to identify brain voxels from non-brain voxels, we use three parallel pathways of 2D convolutional layers in three different planes (i.e. axial, coronal and sagittal planes); third, we propose an auto-context network architecture that significantly boosts sensitivity in the expense of only a slight decrease in specificity, thus generates accurate brain masks. We discuss the details of our proposed architecture in this paper.
Experimental results in this study show that our method outperformed PCNN and the four widely-used publicly-available brain extraction techniques reviewed above on two benchmark datasets (i.e. LPBA40 and OASIS, described in Section 4.1). On these datasets we achieved significantly higher Dice coefficients by our auto-context CNN-based method, named ACNN, compared to the routinely-used techniques. We also examined the performance of ACNN in the challenging problem of extracting fetal brain from reconstructed fetal brain MRI. In this case we only compared our results to BET and 3dSkullStrip as the other methods were not designed to work with the nonstandard orientation and geometry of the fetal brain in MRI. We present the methods in the next section and follow with the experimental results in Section IV and discussion in Section V.

III. METHOD A. Network Architecture
The proposed network has 9 types of input features and 9 corresponding pathways which are merged in two levels. Each pathway contains 3 convolutional layers. This architecture segments a 3D image voxel-by-voxel. For all voxels in the 3D image 3 sets of in-plane patches in axial, coronal, and sagittal planes are used. Each set contains three patches with window sizes of 15 × 15, 25 × 25 and 51 × 51. By using these sets of patches with different window size, both local and global features of each voxel are considered during training. Network parameters are learned simultaneously based on orthogonalplane inputs, so 3D features are learned without using 3D convolution which is computationally expensive. Figure 1(a) shows the schematic architecture of the parallel 2D pathways for one of the 2D views. In the first layer, twenty four 5 × 5 kernels for the patches of size 15 × 15 and 25 × 25, and 7 × 7 kernels for the patches of size 51 × 51 are used. After the first convolutional layer, Relu nonlinear function, max pooling with 2 × 2 kernel size, and batch normalization is applied. For the second convolutional layer, Relu nonlinear function is used after applying convolutional layer with 32 convolutional kernels of sizes 3 × 3, 3 × 3 and 5 × 5, for each patch, respectively. In this layer max pooling is only applied to the two largest patches. In the last convolutional layer 48 kernels of size 3 × 3 are used. After applying Relu function, max pooling is only performed for the largest patch. The output of the third convolutional layer is connected to a fully connected layer with 256 nodes, with 0.5 dropout for each patch. Then, the nodes for each patch are concatenated and connected to a fully connected layer with 256 nodes combined by 0.5 dropout. Each of these 2D pathways collects the information of a 2D plane.
To combine the information of 2D planes, the outputs of the fully connected layers for each set of in-plane patches are concatenated. This results in 768 nodes in total. These concatenated nodes are subsequently connected to the softmax output layer with 2 nodes (brain and non-brain classes). Figure 1(b) illustrates this step. Based on this architecture, we aim to combine low-level features from patches with context information learned by the network to significantly improve classification accuracy. To this end, we propose an autocontext convolutional neural network by adopting the autocontext algorithm designed in [19].

B. Auto-Context CNN
Assuming m training MRI image pairs {(X j , Y j ), j = 1...m}, we add the reshaped 3D input MRI data into the 1D vector X j = (x j1 , x j2 , ..., x jn ) and their corresponding labels into the vector Y j = (y j1 , y j2 , ..., y jn ); where y ji is the where N i is the set of patches around voxel i, i = 1, ..., n, and c is the number of classes (l = 0, ..., c − 1). During the optimization, the cross-entropy between the true distribution q and the estimated distribution p, i.e. H(q, p) = − i q(y i )log p(y i |X(N i )), is minimized. The true distribution follows the Dirac function, i.e. q(y i ) is 1 for the true label and 0 otherwise. The cost function, therefore, would be: In auto-context CNN, a sequence of classifiers is designed in a way that to train each classifier the posterior probabilities computed by the previous classifier are used as features. More specifically, for each image at step t the pair of  (1); Calculate H t using (2); I = |H t − H t−1 |; until I < ; Algorithm 1: The auto-context CNN algorithm x is the output of the first layer for one of the kernels. For the convolution operation with kernel size k we have where W is a k ×k ×d weight matrix, * is the 2D convolution operation, d is the depth of the input feature which is 2, and b is the bias. Expanding the summation in equation (3) we have where W 1 and W 2 are k × k weight matrices corresponding to the intensity input (y 1 ) and label input (y 2 ), respectively. W 2 values are optimized such that they encode information regarding the shape of the brain labels, their respective location, and the connectedness of the labels. During the training of the network at step 0, the weights corresponding to the label input, W 2 , are assigned much lower values than the weights corresponding to the intensity input (i.e. W 2 << W 1 ) since the label input carries no information about the image at the beginning. Note that p 0 j (N i ) is constructed with uniform distribution over classes. On the other hand, in the next steps, the weights corresponding to the label input, W 2 , are assigned higher values than the weights corresponding to the intensity input (i.e. W 2 > W 1 ). Consequently, in testing, the filters corresponding to the predicted labels are more effective than the filters corresponding to intensities.

C. Training
MRI image labels are often unbalanced. For brain extraction the number of non-brain voxels is in average roughly ten times more than the number of brain voxels. The following process was used to balance the training samples: for each training image, 15000 voxels were randomly selected such that 50% of the training voxels were among border voxels. The voxels which had 2 different class labels in a cube of 5 voxels around them were considered border voxels. Of the remaining 50% of samples, 25% were chosen randomly from the brain class and 25% were chosen from the non-brain class.
For training, the cross-entropy loss function was minimized using ADAM optimizer [20]. Three different learning rates were employed during the training. In the first step, the network was trained using a learning rate of 0.001, with 5000 samples for each MRI data pair and 15 epochs. In the second step, learning rate of 0.0001 was used to update the network parameters with another 5000 samples for each MRI data and 15 epochs. Finally, the last 5000 samples for each MRI data and learning rate of 0.00005 were used to update the network parameters.
Figure1c illustrates the procedure of using algorithm 1. To create patches for each voxel in the network, two sets of features are used; first, patches of different sizes around each voxel are considered as inputs, i.e. X(N i ). Second, exact same patch windows are considered around the posterior probability maps calculated in the previous step, p t−1 j (N i ), as additional sets of inputs. The posterior probabilities are multiplied to the mean of the data intensity to be comparable with data intensities. Concatenating these two 2D features provides 3D inputs to the network in two different domains.
Training was stopped when it reached convergence, i.e. when the change in the cross-entropy cost function became asymptotically smaller than a predefined threshold : For testing, the auto-context algorithm was used with 2 steps, i.e. T = 2. Post-processing of generated brain masks involves Gaussian smoothing with standard deviation of 0.5 followed by applying the connected components algorithm on the brain mask and its inverse [21]. This warrants a smooth and fully connected brain mask as is the case for most brain extraction tools.

IV. EXPERIMENTS A. DataSets
We evaluated our algorithm first on two publicly available benchmark datasets and then on fetal MRI data which exhibits specific challenges such as non-standard, arbitrary geometry and orientation of the fetal brain, and the variability of structures and features that surround the brain. We used two-fold cross-validation in all experiments. The output of all algorithms was evaluated against the ground truth which was available for the benchmark datasets and was manually obtained prior to this study for the fetal MRIs.
The first dataset came from the LONI Probabilistic Brain Atlas Project (LPBA40) [22]. This dataset consisted of 40 T1weighted MRI scans of healthy subjects with spatial resolution of 0.86 × 1.5 × 0.86 mm. The second dataset involved the first two disks of the Open Access Series of Imaging Studies (OASIS) [23]. This consisted of seventy seven 1 × 1 × 1 mm T1-weighted MRI scans of healthy subjects and subjects with Alzheimer's disease.
The third dataset contained 75 reconstructed T2-weighted fetal MRI scans. Fetal MRI data was obtained from fetuses scanned at a gestational age between 19 and 39 weeks (mean=30.1, stdev=4.6) on 3-Tesla Siemens Skyra scanners with 18-channel body matrix and spine coils. Repeated multiplanar T2-weighted single shot fast spin echo scans were acquired of the moving fetuses, ellipsoidal brain masks defining approximate brain regions and bounding boxes in the brain region were defined in ITKSNAP [24], and the scans were then combined through robust super-resolution volume reconstruction by either of the algorithms developed in [25] or [26] for motion correction and volume reconstruction. Brain masks were manually drawn on the reconstructed images by two experienced segmenters. Manual brain extraction took between 1 to 4 hours per case depending on the age and size of the fetal brain and the quality of the images.

B. Results
To evaluate the performance of the algorithms, Dice overlap coefficient was used to compare the predicted brain mask P with ground truth mask (extracted manually) R. The Dice coefficient was calculated as follow: where T P , F P , and F N are the true positive, false positive, and false negative rates, respectively. We also report the specificity, T N T N +F P , and sensitivity, T P T P +F N , to compare the algorithms. Figure 2 shows the Dice coefficient, sensitivity, and specificity for the different steps of the training session for fetal MRI in the auto-context CNN algorithm. Significant improvement in the Dice coefficient and sensitivity is observed.
Paired t-test was used to compare the results. The Dice coefficient of the proposed algorithm (ACNN) was significantly higher than BET, 3dSkullStrip, and HWA for LPBA40 and OASIS datasets at α threshold of 0.001 (p < 0.001). Compared to Robex, the Dice coefficient of ACNN was significantly higher at α threshold of 0.01 in LPBA dataset but the difference was not significant (p = 0.5) in the OASIS dataset. Moreover, it revealed significant differences (p < 0.001) between the Dice coefficient of the proposed algorithm with BET and 3dSkullStrip in fetal MRI.  Table I shows the results of the proposed method (ACNN) compared to the other methods on the two benchmark datasets. The results for PCNN were taken from [12]. Our proposed algorithm (ACNN) shows the highest Dice coefficient among all methods, with an increase of about 0.5% over the best performing methods in the LPBA40 dataset. This significant boost in performance was achieved by the architecture of this CNN and the auto-context algorithm which, by incorporating local shape context information along with local patches, allowed a significant increase in sensitivity with only slight decrease in specificity.
The main advantage of our CNN-based method was revealed in the fetal MRI application where the fetal brains were arbitrary oriented and surrounded by a variety of non-brain structures. Figure 3 shows an example, and Table II shows the results of the whole brain segmentation on reconstructed fetal MRI. Only ACNN, BET and 3dSkullStrip were included in this comparison as the other methods were not designed to work with arbitrary brain orientation in fetal MRI and thus performed poorly. As expected, ACNN performed much better than the other algorithms in this application. The Dice coefficient of the ACNN was approximately 10% higher than the other techniques, and the sensitivity was higher by more than 20%. In fact, as seen in figure 3, the other two algorithms generated conservative brain masks which resulted in high sensitivity (close to 1) but with very low specificity.
The effect of using the auto-context algorithm can be seen in figure 3; where the first network is the network without auto-context. Three different improvements are observed after using auto-context steps. First, the label of the brain voxels considered as non-brain by the first network in the middle of the brain voxels (i.e. false negatives) are changed to brain voxels (yellow arrows). Second, the very small number of the non-brain voxels considered as brain voxels in the first network (i.e. false positives) are changed to non-brain voxels. Third, the second step (auto-context) slightly pushes the edges of the brain to the outside (cyan arrows). These three improvements result in remarkable improvement in sensitivity at the cost of only a slight decrease in specificity. The result is a significant boost in segmentation accuracy also shown by an increase in the Dice overlap coefficient. Figure 4 shows the box plots of the Dice coefficient, sensitivity, and specificity of the different algorithms on the three datasets. Robex and ACNN worked well on the benchmark datasets. However, Robex could not be used reliably in the fetal dataset because of the geometric assumptions and the use of an atlas. On the other hand, BET and 3dSkullStrip had more relax assumptions thus could be used, albeit with limited accuracy. It should be noted that none of these methods were designed and tested for fetal brain MRI, so it was not expected that they worked well under the conditions of this dataset. Nevertheless, these experiments show the power of our CNNbased method and the advantages of a geometry-independent, registration-free segmentation method.
The influence of the auto-context network is clear in figure 4 when compared to the output of the network in first step (cyan for the result of the first network and blue for the improvement of the results using ACNN). A significant increase in sensitivity at the cost of only slight decrease in specificity was achieved after the auto-context learning steps. Figure 5 shows logarithmic-scale average absolute error heat map of the different algorithms on the LPBA40 dataset in the MNI atlas space [27]. These maps show where most errors occurred for each algorithm, and indicates that the ACNN performed relatively better than the other methods in this dataset.
V. DISCUSSION Our proposed Auto-context convolutional neural network architecture outperformed the recent deep learning method [12] and four widely-used brain extraction techniques that were continuously evolved and improved over the past decade due to the significant demand for accurate and reliable automated brain extraction in the neuroscience and neuroimaging communities.
We achieved the highest Dice coefficients and a good sensitivity-specificity trade-off among the techniques examined in this paper. This was achieved by using multiple patch sizes as well as context information in a new CNN architecture.
The main advantage of the CNN-based methods in general, and our method in particular, over the routinely-used state-ofthe-art methods is that the CNN-based methods are geometryindependent and registration-free. These techniques are therefore generalizable and can be applied to many applications such as the fetal brain MRI examined in this article. Our technique works on 3D medical images but is efficient due to the use of parallel 2D pathways instead of 3D convolutions. ACNN was easily trained on relatively small number of training datasets as the overlapping image patches already provided large amount of data for training.
With ACNN we overcome one of the persisting challenges in fetal brain MRI processing. The extraction of fetal brain from reconstructed fetal MRI, previously required significant amount of work to correct the masks provided by BET or other level set whole brain segmentation techniques [28]. Atlasbased segmentation methods could not be used as they required alignment to an age-matched atlas, that is also difficult due to the arbitrary orientation of the brain and the inclusion of non-brain material in the absence of brain extraction. Rather than being dependent on an image registration process, the ACNN fetal brain extractions work at the voxel level to mask the fetal brains and prepare them for registration to an atlas space for further analysis.  [12]. Figure 3. Predicted masks overlaid on the data for fetal brain MRI; the top images show the improvement of the predicted brain mask in different steps of the proposed algorithm. The bottom center and left images show the predicted brain masks using BET and 3dSkullStrip, respectively. The bottom right image shows the ground truth manual segmentation. As can be seen, fetal brains can be in non-standard arbitrary orientations. Moreover, the fetal head may be surrounded by different tissue or organs such as the amniotic fluid, other fetal body organs such as hands or feet, umbilical cord, or the uterus wall or placenta. Despite the challenges raised, our method (ACNN) performed very well and much better than the other methods in this application. The Dice coefficient, sensitivity, and specificity, calculated based on the ground truth for this case, are shown underneath each image in this figure. In comparison with other methods, in CNN-based methods, the features are learnt during the training step and no handcrafted features are needed. We used one modality in this study. It is expected that if multiple modalities, such as T1-weighted, T2-weighted, FLAIR or even CT images, are available and used, they result in increased accuracy. The only change in the architecture will be the additional third dimension of the kernel of the first convolutional layer.

VI. CONCLUSION
An auto-context convolutional neural network with three parallel 2D pathways was developed for whole-brain segmentation in 3D magnetic resonance images. This novel technique outperformed a recent deep learning method and four widely-used brain extraction methods in two publicly available benchmark datasets and in the very challenging problem of extracting fetal brain from reconstructed fetal MRI. Unlike the current highly evolved brain extraction methods that use a combination of surface models, surface evolutions, and edge and intensity features, CNN-based methods do not use image registration or assume predefined geometric features such as certain orientations. We achieved superior performance without making such assumptions.
APPENDIX A Theorem 1. The cross-entropy cost function in algorithm 1 monotonically decreases during the training.
Proof. To show that the cross-entropy cost function decreases monotonically we show that the cost at each level will be smaller or at least equal to the cost at previous level. At the arbitrary step t, and Figure 4. Evaluation scores (Dice, sensitivity, and specificity) for three data sets (LPBA40, OASIS, and fetal MRI). Median is displayed in boxplots; blue crosses represent outliers outside 1.5 times the interquartile range of the upper and lower quartiles, respectively. For the fetal dataset the registration-based algorithms were removed due to their poor performance. Those algorithms were not meant to work for images of this kind with non-standard geometry.
Overall, these results show that our method (ACNN) made a very good trade-off between sensitivity and specificity and generated the highest Dice coefficients among all methods including the PCNN [12]. The performance of ACNN was consistently superior in the fetal MRI application where the other methods performed poorly due to the non-standard image geometry and features. Figure 5. Logarithmic-scale absolute error maps of brain extraction obtained from five algorithms on the LPBA40 dataset. This analysis shows that ACNN performed better than the other methods in this dataset.
Also, note that the posterior probability is: p (t) (y i = k|X(N (i), p t−1 (N i )) = e fy k (X(Ni),p t−1 (Ni)) c e fy c (X(Ni),p t−1 (Ni)) (9) Using f y k (X(N i ), p t−1 (N i )) = log p (t−1) i (y i ) cross-entropy in level t will be equal to cross-entropy in level t − 1. Since, during the training in step t we are minimizing the cross entropy cost function, p (t) i (y i ) should at least work better than p (t−1) i (y i ). Therefore: ACKNOWLEDGMENT This study was supported in part by the National Institute of Biomedical Imaging and Bioengineering of the National Institutes of Health (NIH) grant R01 EB018988. The content of this work is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.