{'fraction_non_alphanumeric': 0.027981491182119782, 'fraction_numerical': 0.00890518596123625, 'mean_word_length': 4.703012198157829, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Automatic Recognition of Anatomical Regions in Computed Tomography Images

This paper presents a method that can recognize anatomy regions in Computed Tomography (CT) examinations. In this work the human body is divided into eleven regions from the foot to the head. The proposed method consists of two main parts. In the first step, a Convolutional Neural Network (CNN) is used to classify the axial slices of the CT exam. The accuracy of the initial classification is 93.4 %. As the neural network processes the axial slices independently from each other, no spatial coherence is guaranteed. To ensure the contentious labeling the initial classification step is followed by a post-processing method that incorporates the expected order and size of the anatomical regions to improve the labeling. In this way, the accuracy is increased to 94.0 %, the confusion of non-neighboring regions dropped from 1.5 % to 0.0 %. This means that a continuous and outlier free labeling is obtained. The method was trained on a set of 320 CT exams and evaluated on another set of 160 cases.


Introduction
Three-dimensional medical imaging techniques are routinely applied in the clinical practice. Computed Tomography (CT) as well as Magnetic Resonance Imaging (MRI) are integral parts of diagnosis, therapy planning, and monitoring. As the number of the medical images is continuously increasing [1], the computer aided processing of these images becomes more and more important.
To provide support for the therapists several algorithms have been developed for various clinical workflows, like segmentation, registration or visualization. These methods are usually specialized to a specific region of interest (ROI). There are different applications for lung nodule detection [2], virtual colonoscopy [3], cardiac or cerebral vessel analysis [4], tumor follow-up [5,6] etc.
The DICOM standard involves tags to specify the anatomy location for each slice of an image series, but it is rarely filled in by the scanner and the manual specification is time consuming and error prone [7]. Automated pre-processing functions cannot rely on this information, which can lead to incomplete processing of medical images that requires time consuming correction from the user.
From the above-mentioned examples one can see that the automated detection of anatomical regions in medical images would have great impact to the content-based medical image processing. It would create many opportunities to automate or optimize various types of algorithms (i.e. initialization of segmentation or registration methods), which would save significant amount of time and workload for the clinicians. Although the need for this function does not directly come from the user, its benefits are definitely welcome in medical image processing.
In this work, we present a method that can reliably recognize the main anatomical regions in 3D CT images. The recognition process starts with an image-preprocessing step and a preliminary recognition process that utilizes convolutional neural networks. These standard steps are followed by a newly introduced post-processing method that can filter the initial classification results to ensure a correct final labeling.

Related Work
Over the last few years, deep learning methods became popular in the computer vision community. The convolutional neural networks (CNN) are especially designed to work with image processing tasks (object detection, classification or segmentation), as images contain highly correlated local intensity patterns. The CNN methods [8][9][10] regularly outperform the traditional image feature extraction based methods in image classification tasks.
As the automatic classification of human anatomy is an important prerequisite in many computer aided diagnostic processes, many researchers aimed different level of recognition of the anatomical regions. There were approaches to identify anatomic landmark points [11][12][13]. These methods provided good results in case of full body CT scans where all the required landmarks are visible.
Other techniques were developed to classify the axial slices of scans using a manifold learning method [14,15]. The reported accuracy is around 94.0% considering six classes.
The recent VISCERAL challenges [1,16] have also showed that the recognition of body parts is a very active field of research and could be well utilized in clinical practice.
CNNs were also applied for recognition of anatomical regions. Yan et al. [17] provided a great overview about the application of different CNN configurations in the classification of axial images. Roth et al. [18] used five classes to identify axial slices of CT images and achieved the stateof-the-art accuracy (5.9 % error).
In summary, the classification of the axial slices of CT images is widely being researched. The recent CNN classification methods provide state-of-the-art quality results. However, the CNNs use statistical models and therefore can produce noisy output, which cannot be directly used in the clinical practice. It is worth incorporating a priori information of the human body to stabilize the results of the CNN and to provide a reliable classification.

The Data Set
The CT examinations used in this work (for training as well as evaluation) were originally collected to evaluate various types of clinical applications in the area of oncology, cardiology, surgery, neurology. The cases involve wholebody as well as partial scans. The images were acquired with whole (50-70 cm) as well as small (20-35 cm) reconstruction diameter. The dataset shows significant variance in patient's sex, age, and level of obesity. It includes native and contrast-enhanced cases, and exams demonstrating various types of abnormality (pathology, implant, noise). Thus, it is a representative set of 480 CT images.
The definition of anatomy regions: The primary goal of this work was to facilitate automated processing of CT images. The definition of the anatomy regions was driven by the needs of organ segmentation algorithms (e.g. brain, lung, liver, colon, prostate). Based on this consideration eleven anatomy regions were defined in agreement with radiologists as defined in Table 1.
Each region starts at the axial slice involving the starting point and ends before the first slice of the next region. The above defined anatomy landmarks were manually defined by radiologists for all examinations, and each slice in the dataset was automatically labeled with the corresponding region label based on its location. Table 2 summarizes the statistical properties of the established regions. This information is later utilized in Section 8.

Method
The proposed method consists of three main steps. In the first step, traditional image processing methods are used to normalize the axial slices of the CT scan. With this step the most interfering artifacts can be eliminated and the slices can be used in a uniform way in the further processing steps. The next stage of the method is the initial classification of the axial slices. In this step, a CNN is used to assign anatomy labels to the slices. In addition to the most likely label, the confidence for each label can also be obtained. These confidences play important role in the post-processing step.
In the last phase, the initial labeling and the region confidences are incorporated. The output of the CNN can be noisy and can contain classification errors, as it is a result of a statistical model. To obtain a result that is usable in clinical practice, some properties of the final labeling should be ensured. This final step guarantees that the order of the detected anatomical regions is correct and all the detected regions have a realistic size.

Image Processing
As the imaging options can vary from scan to scan, a pre-processing step is applied before slice classification.
The position of the patient with respect to image center can vary significantly among examinations, which introduces unwanted variation. Even if the patient is acquired in supine position the table can be lifted in anterior or posterior direction. In order to compensate that, the weight center of non-air voxels is computed that is used to define the body axis for the whole CT exam.
After this point each slice is separately processed. First, a squared region is extracted, such that its center is located on the body center axis (that is defined as the average of the weight center for all slices), its size (both width and height) is equal to 35 cm, and its resolution is equal to 256x256 pixels. Since the pixel size of CT images varies among examinations interpolation is used. If the original input slice covers larger or smaller image region than 35 cm, the pre-processed image is cropped or padded (with air voxels), respectively. Then, the pixel intensities (from the original 16-bit signed integer value) are rescaled to the range [0,255], such that the range [-500,500] HU is linearly mapped to [0,255]. This step enhances those tissue types which are key important from anatomy point of view and prevents the classifier taking very high or low density pixels into account. Fig. 1 shows the result of the pre-process. In consequence of this step the most interfering imaging options can be normalized and the resulted axial slices can be used in a uniform way in the following steps.

Convolutional Neural Networks
CNNs have been proposed in [19] to classify 2D image data. These types of neural networks consist of several convolutional and sub-sampling layers followed by fully connected layers. An example is shown in Fig. 2. As Ravi [20] summarized, the main concepts can be expressed in three points: • The input image is convolved using several small filters. • The output of the previous step is sub-sampled. • The output of the sub-sampling is considered as a new image and the convolution and the sub-sampling process is repeated until high level features are extracted.
In this way low level image features are extracted first, then by combining these low level features higher level image descriptors are obtained in the next layer. Repeating these steps, the required information is extracted from the input image to perform the classification that is the task of the last fully connected layer. The output of the last layer can be directly used to assign labels for each axial slice and to get the confidence vector of the labeling.
In this work AlexNet [8] was used for the classification of the 2D grayscale images. The structure of the NET follows the originally introduced topology.
The image dataset was split to three nearly equal subsets, such that the training, the cross-validation, and the test  sets included 66 000, 55 000, and 60 000 labeled images, respectively. The first set was used to train the classifier, the second was used to monitor the accuracy during the training, and the last one was used to evaluate the CNN model (as well as the post-processed result). In order to simulate all possible patient positions, the axial slices of the training dataset (that included images acquired in supine position only) were rotated 0, 90, 180, and 270 degrees. The training was performed within Nvidia DIGITS framework [21] using the Caffe [22] deep learning toolkit. Stochastic Gradient Descent method was applied as a solver. 30 training epochs were performed. The learning rate was set to 0.01 at the beginning and it was divided by 10 after 10 and 20 epochs. It took 5 hours to train the CNN using a simple (Nvidia Quadro K4000) GPU.

Initial Classification
In this section the initial classification results are presented to illustrate the capabilities of the CNN, to highlight its limitations and to establish the necessity of the proposed post-processing method. Table 3 presents the confusion matrix of the applied CNN network evaluated on the test data set. The columns of the table specify the known ground truth labeling while the rows represent the labeling assigned by the trained system.
For each label the Precision value was calculated as the number of the correctly labeled slices divided by the number of all slices that were associated with that specific label by the CNN. The Recall metric is defined as the number of the correctly labeled slices for a specific label divided by the total number of slices belonging to that specific class according to the ground truth labeling. The overall accuracy (the last cell in the last row) is defined as the ratio of the number of all correctly labeled slices to the total number of slices in all classes.
The overall accuracy of the initial classification was 93.4 % and the precision varied between 85.1 % and 97.4 %. The values in the main diagonal represent the correctly labeled slices. The values directly below or above the main diagonal indicate some minor confusion, usually occurring at the border of the anatomical regions. For example 143 HEAD slices were labeled as CHEST. Other confusions mean serious misclassification errors. This effect indicates the independent processing of the slices, as the labeling process does not incorporate the neighborhood information at this point.
Although the majority of the slices were correctly labeled by the CNN there are some major artifacts that makes the result impractical for direct clinical application. Fig. 3 shows two examples from the test set. In the images the central coronal slice of the CT exams are shown and each row of the images represent an axial slice of the corresponding CT exam. In Fig. 3(a) the initial classification accuracy is 84.7 % and it is free from significant misclassifications while Fig. 3(b) is one of the worst initial results (accuracy: 67.8 %) and contains examples for all major errors (these errors may caused by the non-supine positioning during the acquisition). The left side of the images show the initial labeling (based on the maximal confidence values) and the confidence values are plotted on the right side. The different colors indicate different anatomical regions (yellow -BRAIN, olive -HEAD, yellow-green -CHEST, dark green -ABDOMEN-UPPER, green -ABDOMEN-LOWER, dark magenta -PELVIS-UPPER, red -PELVIS-CENTER, magenta -PELVIS-LOWER, dark blue -THIGH, middle blue -SHIN, blue -FOOT). As the labeling of the slices are independent, the CNN can produce alternating labeling near the region borders (e.g. between PELVIS-CENTER and PELVIS-UPPER) or can completely miss the labeling as can be seen in the HEAD and CHEST regions (see Fig. 3(b)). This confusion of non-neighboring regions are major classification errors and affect 1.5 % of the slices of the whole test set.

Post-processing
To eliminate the above mentioned artifacts of the initial classification, a post-process is introduced. The main goal of this step is to produce a continuous and reliable labeling for the whole image series. A labeling can be considered acceptable if it meets the following requirements: • The labeling must be continuous. No alternating labeling can occur. • The anatomical regions should appear in their correct order e.g. the CHEST region should be followed by the HEAD • The size of the anatomical regions should be reasonable. The regions cannot be arbitrary small or large.
To obtain the desired properties of the labeling a region membership function fitting algorithm is proposed. In this method the anatomical regions are represented with generalized normal distributions (GDF) [23] as defined in Eq. (1), where μ is the location, α is the scale, β is the shape parameter and Γ(x) is the gamma function. Examples of this function can be seen in Fig. 4.
Each membership function can be customized to have region specific parameters. To estimate the position (μ) and range of the membership functions (α), the confidence values, provided by the CNN, can be utilized. The β shape parameter is set to a fixed value (β = 6) in this work.
In the first step of the method, a set of regions, which are possibly involved in the image, is selected. This provides a list of candidate regions. From this list the anatomically correct sequences are assembled and the most likely one is chosen for further processing.
In the next step, a membership function set is produced to represent the selected region sequence. Based on the confidence values of the initial labeling the position (β) and range (α) parameters of the functions are optimized. In this way the functions are fit on the image series and a realistic final labeling can be obtained.

Selecting the Most Likely Region Combination
To estimate the visible regions in the image, the confidence weighted size of each region S i is calculated as described in Eq. (2): where N is the number of the slices, conf (k, i) is the confidence value provided by the CNN describing the likelihood of that slice k is located in region i, and T k is the physical thickness of slice k. If S i is greater than a predefined limit, the region is considered visible in the image. In this way the small or misclassified regions can be eliminated from the further processing.
As soon as the visible regions are available, all of the anatomically correct sequences of these regions are assembled. To select the most likely sequence the Summarized Accepted Confidences (SAC) is calculated in Eq. (3): where M is the number of regions in the sequence, Rc i − 1 and Rc i + 1 are the confidence weighted center of the previous and next regions, respectively. Rc k Rc is 1 if k is between Rc i − 1 and Rc i + 1 otherwise it is 0 . This means the confidence value conf (k, i) is accepted only if the position of the k-th slice is between the centers of the neighboring regions. This eliminates the most interfering outliers from the SAC calculations. This way the SAC value indicates the summarized confidence values of the slices that are positioned and labeled correctly according to the processed sequence.
Once all the SAC values are calculated, the region sequence with the highest SAC can be selected as the most likely combination of the candidate regions.

Fitting the Region Membership Functions
After selecting the most likely region sequence, a set of membership functions is fitted on the confidence values. This process estimates the position and range parameters of the functions to provide a continuous and reliable labeling. Regarding these calculations, the following considerations are taken: • The membership functions should maximize the correctly covered confidence values. To perform the parameter estimation an iterative method is used. The position of the membership functions ( μ ) are initialized with the confidence weighted center of each region ( Rc i ) and their range property ( α ) is set to the average size of the represented region (see Table 2). Fig. 4(a) shows the initial position and range estimations for a given confidence distribution.
To meet the requirements, a cost function is defined that consists of three components. The first one, defined in Eq. (4), penalizes if a membership function ( GDF j ) does not cover slices that belong to region j .
The second component Eq. (5) penalizes if a membership function covers slices that do not belong to that specific region.
The third component of the cost function Eq. (6) tries to minimize the overlap between the region membership functions.
The final cost function is defined as the sum of the three components.
To perform the optimization and to find the desired values for the α and μ parameters of the region membership functions a gradient descent iterative optimization method is applied. Fig. 4(b) shows the membership functions after the post-process. It can be observed that the functions are positioned according to the input confidence values and their range fit the confidences as well. The correct ordering of the regions is maintained during the iterations and their size is in the acceptable range.

Results
Using the post-process, the artifacts of the initial labeling can be eliminated and a continuous and reliable labeling can be obtained. The post-processed results are summarized in Table 4. As one can see the overall accuracy increased to 94.0 %. Furthermore, only the main diagonal and values below and above that are not zero, so the most interfering errors, confusion of non-neighboring regions, were completely eliminated thanks to the post-process. As the post-process guarantees the continuous labeling, the values directly below and above the main diagonal of the confusion matrix mean only small displacements of the borders between the neighboring regions.
To illustrate the effect of the post-process Fig. 5(a) is presented. Compared to Fig. 3(b) one can see, that the initial misclassification errors are eliminated and a continuous labeling is obtained without alternating labels near to the borders of the regions. Fig. 5(b) shows the ground truth labeling of the examined CT series. It can be observed that the order of the anatomical regions is correct, most of the regions have the correct size and only the PELVIS-LOWER region became smaller than the original one in the ground truth image.
In Fig. 6 other series can be seen to demonstrate the accuracy of the presented method. It can be observed that the method works well in challenging cases. Fig. 6(a) shows a patient with high level of obesity and the image has an artifact in the right arm, Fig. 6(b) is a long leg section, in Fig. 6(c) a large tumor can be seen in the left lung, Fig. 6(d)and (e) are partial body scans that are frequently acquired in the clinical practice while Fig. 6(e) shows a scan with non-supine patient position during the acquisition.
From the above mentioned examples one can see that the presented method can handle wide range of clinical cases reliably and can produce continuous labeling.
The required time to perform the calculations is composed of two factors. The initial slice classification takes a few (2-3) seconds to run using GPU, while the post-processing step takes an additional two seconds to run. In summary, the presented method takes four to five seconds to process an average CT examination.

Conclusion
In conclusion, this paper presents a method that detects anatomy regions in CT images. The proposed approach applies deep-learning to classify 2D axial slices of the image followed by a post processing step that incorporates a-priori anatomy information to improve the accuracy. The major advantages of the method is that it recognizes all anatomy regions, it can process whole-body as well as partial scans, and it does not require the whole cross-section of the body to be involved in the image. The evaluation shows that the proposed technique provides accurate results on wide range of clinical CT cases, so it can be a good basis of an automated image processing system. In future work the method can be extended to other 3D imaging modalities such as Magnetic Resonance.