{'fraction_non_alphanumeric': 0.02173100289746705, 'fraction_numerical': 0.008972801196373493, 'mean_word_length': 4.7944760357432985, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A 3D fully convolutional neural network and a random walker to segment the esophagus in CT

Precise delineation of organs at risk (OAR) is a crucial task in radiotherapy treatment planning, which aims at delivering high dose to the tumour while sparing healthy tissues. In recent years algorithms showed high performance and the possibility to automate this task for many OAR. However, for some OAR precise delineation remains challenging. The esophagus with a versatile shape and poor contrast is among these structures. To tackle these issues we propose a 3D fully (convolutional neural network (CNN) driven random walk (RW) approach to automatically segment the esophagus on CT. First, a soft probability map is generated by the CNN. Then an active contour model (ACM) is fitted on the probability map to get a first estimation of the center line. The outputs of the CNN and ACM are then used in addition to CT Hounsfield values to drive the RW. Evaluation and training was done on 50 CTs with peer reviewed esophagus contours. Results were assessed regarding spatial overlap and shape similarities. The generated contours showed a mean Dice coefficient of 0.76, an average symmetric square distance of 1.36 mm and an average Hausdorff distance of 11.68 compared to the reference. These figures translate into a very good agreement with the reference contours and an increase in accuracy compared to other methods. We show that by employing a CNN accurate estimations of esophagus location can be obtained and refined by a post processing RW step. One of the main advantages compared to previous methods is that our network performs convolutions in a 3D manner, fully exploiting the 3D spatial context and performing an efficient and precise volume-wise prediction. The whole segmentation process is fully automatic and yields esophagus delineations in very good agreement with the used gold standard, showing that it can compete with previously published methods.


I. INTRODUCTION
Precise delineation of organs at risk (OAR) is a crucial task in radiotherapy treatment planning (RTP), which aims at delivering high dose to the tumour while sparing healthy tissues. While some OAR presenting high contrast to neighbouring structures are easy to depict in CT scans, the esophagus is sometimes very difficult to demarcate from the mediastinal structures. On some CT slices even experts have difficulties to define reliably its boundaries and it can easily be confused with other structures. This leads to a tedious interpretation of the CT images, which makes the process not only time-consuming, but also highly prone to inter-observer variability 1 . However, with its radio-sensitive mucosa the esophagus is one of the most critical OAR in RTP, and a reliable contouring is indispensable.
Thus, automatic, reproducible and reliable segmentation approaches of the esophagus are greatly desired to be adopted in the RTP.
Although considerable attention has been devoted to automating the process in some other structures in CT 2 , literature on esophagus segmentation remains very rare with results far from being fully satisfactory. Indeed, presented approaches which successfully segmented a variety of structures on CT often failed to segment the esophagus 3,4 . One of the main reasons is the absence of consistent intensity contrast between the esophagus and surrounding tissue in thoracic CT scans. Additionally, its appearance varies depending on whether it is filled with air, remains of orally given contrast agent, or both. Furthermore, the esophagus shows a certain mobility. Thus, the esophagus can be considered as a structure with very inhomogeneus appearance and complex shape. Even though some attempts to segment the esophagus have been proposed, most present three main drawbacks: first, they require several levels of user interactivity (placing manual points 5,6 , manual region selection 7 or drawing contours at some slices 8 , for example); second, some of them assume that contours of surrounding organs are available 5,9 ; and third, registration steps are required in most cases, which may introduce an additional source of error. Despite all these efforts, the automation of this task still remains very challenging, achieving low performance in most cases.
On the other hand, deep learning has recently emerged as a powerful classification tool, achieving state-of-the art performance in numerous applications of pattern or speech recognition. Among the different types of deep learning approaches, convolutional neural net-4 works (CNNs) 10 have shown the greatest potential for computer vision and image analysis problems. Particularly in medical image segmentation, CNNs are becoming very popular, achieving outstanding performance in various applications 11 . These architectures are supervised models that are trained end-to-end to learn a hierarchy of features representing different levels of abstraction. In contrast to separate hand-crafted features and classifier models, CNNs can learn simultaneously image features and the classifier. This type of architectures are typically made up of multiple convolution, pooling and fully-connected layers, which parameters are learned using back-propagation by minimizing a given cost function.
The pixel-wise segmentation classes in standard CNN approaches are predicted independently, based on its enclosing region, which generates non-spatial outputs. By reinterpreting classification layers as fully convolutional layers, a CNN network is able to generate spatial output maps for all the classes, becoming a natural choice for dense problems, such as semantic segmentation. These architectures, which are known as fully convolutional neural networks (FCNNs) 12 , can be seen as a large non-linear filter whose output yields class probabilities. Therefore these networks can accommodate images of arbitrary size and provide much greater efficiency by avoiding redundant convolutions/pooling operations. However, although medical images are commonly presented as 3D images, most of the existing CNN approaches perform the segmentation in a slice-by-slice fashion, not fully exploiting 3D context. Even though the extension of 2D convolutions to 3D seems natural, the additional computational complexity introduces significant challenges, making that 3D CNNs have been mostly avoided up to recent times. Nevertheless, an important benefit of 3D convolutions over 2D representations is their ability to fully exploit dense inference 13 , which allows to decreasing inference times. Recently, 3D FCNNs yielded outstanding segmentation performances in the context of brain lesions 14 and subcortical brain structures 15 .
Nevertheless, if there is a lack of annotated massive data and/or a high complexity of the problem at hand, CNN architectures tend to produce imprecise segmentations. To further improve segmentation results, some authors have interpreted the CNN output as potentials of a Markov Random Field 16 or as initialization to drive a level set segmentation 17 , for example. In this way, coarse pixel (or voxel) level label predictions from the CNN are refined to produce sharp boundaries and fine-grained segmentations.
Contributions: In this study, we explored the feasibility of guiding a random walk (RW) algorithm 18 with a CNN and its application to CT esophagus segmentation. Particularly, we trained a CNN to generate an esophagus probability map, which was used to drive a RW segmentation. We believe this is the first work to address esophagus segmentation by employing deep learning. Additionally, and to the best of our knowledge, this work also represents the first attempt to drive a RW by employing CNN-based probability maps. To generate these probability maps, we employed a 3D fully CNN, which use in the medical field has been very limited up to date. With the use of 3D context, our method can handle volumetric homogeneity better than 2D-based convolutions, which is particularly important in thin tubular structures. Furthermore, most of the existing works address the problem by requiring some sort of user interaction. Contrariwise, our proposed method performs the task in a fully automated manner. And last, we report state-of-the-art performance in one public and in one clinical dataset, showing a good agreement with clinically used peer reviewed esophagus annotations.

II. MATERIALS AND METHODS
The proposed segmentation approach is three fold. First, a probability map of the esophagus is generated with a 3D fully CNN. Then, an active contour model (ACM) is fitted on the probability map for a first estimation of the esophageal centerline. Finally, probability map and centerline are used in combination with Hounsfield Unit and image gradient information to drive the RW towards the final contour. The architecture of our segmenation approach is depicted in Fig. 1 a. For evaluation, we run extensive experiments on a publicly available dataset as well as on clinical CTs used in RTP.  fully-connected layers are added at the end of the network to encode semantic information.

II.A. Prior semantic esophagus segmentation
Particularly for this work, three fully-connected layers, composed of 400, 200 and 150 hidden units were added. To ensure that the network contains only convolutional layers, we use the strategy described in 14 and 15 , in which fully-connected layers are converted to a large set of 1×1×1 convolutions. By doing this the network can retain spatial information and learn the parameters of these layers as in other convolutional layers. A down-sampling factor of 2 along the three dimensions is employed in the second path of the architecture (see 14 for a detailed explanation of the dual-path approach followed) to capture a larger receptive field, which leads a broader view of the context. In the dual path both number and size of kernels is the same than in the main path. Due to computation and memory limitations, our network can not apply dense training over the whole 3D input volume. Instead, volumes are split into S smaller sub-volumes, allowing dense inference on these samples.
A stride may also be defined for each convolutional layer, representing the displacement of the filter, along the three dimensions, after each application. To preserve spatial resolution, we use a unit stride for all convolutional layers. Each convolutional layer is followed by a Parametric Rectified Linear Unit (PReLU) 19 , which applies an element-wise activation function. Let's define θ as the network trainable parameters (i.e., convolution filter weights, biases and PReLU activations in our case), and denote as L the set of ground-truth labels such that L v s ∈ L represents the label of voxel v in the s-th sampled sub-volume. Following 15 , we defined the cost function as where p c (X v ) is the output of the classification layer for voxel v and class c (i.e. softmax output). As in 15 we set the sampled sub-volume size in our network to 27×27×27 for training and 45×45×45 for testing. To make the centerline processable for the RW algorithm, it is transformed to a distance map with a value of 1 at the centerline and linearly decreasing values, which reach a minimum 8 of 0 at a distance of 25 mm from the centerline. An example can be seen in Fig. 2d. The radius of 25 mm was chosen as it generously covers the usual width of the esophagus (usually 15 to 20 mm 23 ) but excludes other unwanted structures.

II.C. Random Walk
To generate a final contour on basis of the CNN generated probability map, the ACM distance map and the CT image data a random walk algorithm (RW) incorporating prior models 18 was chosen. This algorithm was selected because it is able to join information from non spatial data like intensity models and spatial correlations like voxel neighborhoods, and in contrast to the original work 24 needs no input labels. The algorithm constructs a weighted undirected graph. In the graph each voxel is represented by a vertex and connected to its direct adjacent neighbors by a weighted edge. Additionally, for every segmentation class (esophagus and non esophagus) a vertex is added to the graph and connected to all voxel vertices (see Fig. 1 b). With the use of the Laplace matrix of the graph, a system of linear equations can be created and solved to calculate the probabilities for all voxels to belong to a given class. A detailed mathematical derivation can be found in 18 and 24 .
The proposed RW makes use of CT image gradients and a prior model. Edge weights between adjacent voxel vertices are calculated with the use of the CT image gradients by: where δ ij is the intensity difference between voxels represented by vertex i and j, σ δ and µ δ are standard deviation and mean of the gradients inside the esophagus contours of the learning datasets. Z is a normalization constant to scale the highest value of w(e ij ) to 1 and the smallest value to 0. The prior probability model used incorporates information from the CNN, the ACM and the CT image intensity distribution. For this, two vertices, representing the the two segmentation classes, are added to the graph and connected to all voxel vertices by edges. The weights for the esophagus class and non esophagus class are calculated by: and w non esophagus (e i ) = Nevertheless, volume-based metrics are very insensitive to edge differences when they have a small impact on the overall volume. This makes that two segmentations showing a high degree of spatial overlapping might present clinically relevant differences at the edges. This is particularly important in RTP, where the contours serve as critical input to compute the delivered dose. An analysis on shape fidelity of the segmentation outline is therefore highly recommended, since any underinclusion on the OAR segmentation might lead to a part of the healthy tissue exposed to radiation. Thus, distance-based metrics -Hausdorff distance (HD) and average symmetric surface distance (ASSD) -were also employed.

III. RESULTS
Evaluation was done in three steps. First, the benefit of the RW step was inspected. For this the RW contour was compared to the 50 % threshold contour of the CNN probability map and a morphologically closed version of the 50 % contour to exclude small false positive spots. Second, the esophagus contours as a whole were analyzed. In a third step, we cut the datasets below the lower tip of the heart and analyzed only the upper part of the organ.
This was done because for the best performing fully automatic algorithm so far published 29 performance was analyzed only in the region from the thyroid gland down to a level below the left atrium and from our perspective the lower part of the esophagus until the gastroesophageal junction is a very challenging one to delineate, as the esophagus changes its shape in this region (compared to the upper part, the diameter gets larger) and the transfer from esophagus to stomach is hardly visible.
We want to emphasize that a first analysis of the results showed up that probability values generated by the CNN for patient 7 in synapse dataset were close to zero in nearly the whole image. A deeper inspection revealed that this case is the only one presenting intense stripe shaped artifacts (an example is depicted in Fig. 6a). As these artifacts are only visible in one subject, the CNN was not able to learn how to deal with them. Therefore, this subject was excluded in further evaluations.
In  Table I: Mean values of DSC, ASSD and HD for the three configurations analyzed on both datasets. While first part of results represents segmentations on the whole volume, the second part deploys performance on cropped generated contours.    Table III presents the results obtained by the proposed method in both datasets analyzed and puts our approach in context with other methods found in the literature. It can be seen that only one semi automatic approach 5 achieved better performance in terms of volume overlapping. Nevertheless, when considering distance based metrics our algorithm yielded the best results. For example, taking results from both datasets into account, the mean ASSD was reduced nearly 35% and 25% with respect to the best semi-automatic 9 and the fully automatic approach 29 , respectively.
Visual results of our algorithm compared to the reference contour for 6 cases in axial and sagittal view are depicted in Fig. 5. From the sagittal views it can be observed that manual contours are not as smooth as the automatic contours, which could be due to the breathing artifacts. The use of convolutions and post-processing in 3D aids at processing information of the 3D context through several slices, which is reflected in the smoother automatic contours  shown in these images. On the other hand, from the axial views we can realize that, despite of the similarity with respect to neighboring tissues, and heterogeneity of the inner region of the esophagus, our automatic system provides contours that are comparable with the reference standard.
Segmentation was achieved in 90-100 seconds, on average, in a whole CT scan. In particular, CNN probability map calculation took between 40-60 seconds, ACM probability map calculation 30 seconds, CT probability values 7 seconds and generating the final contour by processing all probability maps with the RW 10 seconds on average.

IV. DISCUSSION
We explored the feasibility of driving a random walk (RW) algorithm 18 with a CNN and its application to CT esophagus segmentation. Our results prove that a CNN is able to contour the esophagus almost to a perfect extent in some cases (DSC > 0.9 for clinical cases 09 and 10). However, one can also note that the DSC values for the CNN strongly vary over all cases. Adding a RW for post processing made the performance more homogeneous and, as a whole, increased all figures of merit. The reason why CNN + RW worsened results for some cases is that for those cases the CNN 50 % outline fitted the reference contour already very well and showed very few false positive spots. The RW post processing then enlarged the contour, which resulted in worse figures of merit (an example can be seen in Fig. 6b). The enlargement was due to "jumps" of the depicted esophagus from one slice to the consecutive ones. As the RW algorithm considers the neighborhood of voxels in all directions it enclosed non-esophageal tissue in those "jump" areas. Examples can be seen in Fig. 7 and Fig. 5 a-c.
The "jumps" in the clinical cases arose because patients were freely breathing during CT acquisition. The respiratory motion of the thorax during acquisition caused numerous image artifacts. Some examples are depicted in Fig. 7. Considering the bad image quality due to respiration, emphasizes the good performance of the proposed algorithm even more. Also the big variations in the CNN results without RW post processing may be due to the rather small training datasets and/or to the respiratory motion artifacts. Training and testing the algorithm again on gated or average CTs could probably increase its accuracy.
For public dataset 7 the CNN resulted in a probability map with almost all values close to zero. We suggest that this arises from the strong artifacts (see Fig. 6a) only visible in this case. The stripe shaped artifacts change the texture of all visible organs and hence the results of the convolution filter. As this case is the only one in the dataset with such artifacts the CNN was not able to learn how to classify the differently looking structures correctly.
The origin of the artifact seems to be the arms of the patients lying parallel to the thorax and abdomen in the reconstructed field of view (FOV Our promising results open the perspective to perform further evaluations to investigate the impact of our automatic contours on radiotherapy planning. Published studies showed that esophageal motion during one treatment fraction and between two or more fractions is a non negligibly issue. Due to respiration and cardiac motion intra fractional movement of the esophagus between 5 and 10 mm was reported 30,31 , even though it can be up to 15 mm 31,32 .
Additionally, it was shown that the motion is highly patient and location dependent 31,32 .
Therefore, an expansion and evaluation of our algorithm to 4D time gated CT and cone beam CT is intended. On the one hand to provide an automatic tool to measure patient specific esophageal motion for treatment planning and on the other hand to offer a fast and reliable method to account for inter fractional motion and setup errors to enable adaptive planning. To do that, one option might be to generate segmentations in 3D volumes across different time frames, and apply a conditional random field as a post-processing step to impose consistency in time.

V. CONCLUSION
In this work we present a fully automatic algorithm for contouring the esophagus on CT.
Our algorithm first generates an accurate estimation of the esophagus by employing a CNN.
In a second step the estimation is refined to the final contour by using an active contour model and a random walker. One of the main advantages of our CNN are the 3D convolution kernels, which allow a fully exploitation of the 3D spatial CT information. Compared to other automatic algorithms our method shows superior spatial overlap and similarity in shape with the reference contours.