{'fraction_non_alphanumeric': 0.033285642147786565, 'fraction_numerical': 0.014391516579700387, 'mean_word_length': 5.088905969766846, 'pattern_counts': {'":': 0, '<': 4, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Deep Learning Body Region Classification of MRI and CT examinations

Standardized body region labelling of individual images provides data that can improve human and computer use of medical images. A CNN-based classifier was developed to identify body regions in CT and MRI. 17 CT (18 MRI) body regions covering the entire human body were defined for the classification task. Three retrospective databases were built for the AI model training, validation, and testing, with a balanced distribution of studies per body region. The test databases originated from a different healthcare network. Accuracy, recall and precision of the classifier was evaluated for patient age, patient gender, institution, scanner manufacturer, contrast, slice thickness, MRI sequence, and CT kernel. The data included a retrospective cohort of 2,934 anonymized CT cases (training: 1,804 studies, validation: 602 studies, test: 528 studies) and 3,185 anonymized MRI cases (training: 1,911 studies, validation: 636 studies, test: 638 studies). 27 institutions from primary care hospitals, community hospitals and imaging centers contributed to the test datasets. The data included cases of all genders in equal proportions and subjects aged from a few months old to +90 years old. An image-level prediction accuracy of 91.9% (90.2 - 92.1) for CT, and 94.2% (92.0 - 95.6) for MRI was achieved. The classification results were robust across all body regions and confounding factors. Due to limited data, performance results for subjects under 10 years-old could not be reliably evaluated. We show that deep learning models can classify CT and MRI images by body region including lower and upper extremities with high accuracy.


INTRODUCTION (400 words)
Improved accuracy of anatomic classification of imaging data is required to improve the usability of medical imaging data throughout imaging and enterprise workflows (1). Body region categorization is foundational information in radiology workflows such as ordering, triage, acquisition, interpretation and report communication. Anatomical metadata is used by humans and machines for selection of relevant prior studies, organizing display protocols, registration and navigation of studies, and contextual interpretation and reporting. It is also a pre-requisite for diagnostic AI such as Computer Aided Detection (2) and Computer Aided Triage systems (3) that work for specific anatomies and clinical indications However, in practice, body region metadata documented in DICOM labelling of series and studies is often inconsistent within and across institutions. For example, studies for knee imaging may be labelled as "knee MRI" in one institution and "lower extremity CT" in another institution. Similarly, a "CT elbow" may be called "CT upper extremity" in another institution. As important, the anatomical content of an imaging study cannot always be summarized to one body region.
For example, an abdomen exam may contain images of the chest that may be important to review if a new lung nodule is found on subsequent chest imaging. Accurate labelling of the anatomy covered by a study could therefore have an impact on the management of incidental findings. In short, current inconsistencies in the way anatomical metadata is documented makes the search and access to anatomically matching studies cumbersome and inefficient and has a detrimental impact on a radiologist's accuracy and productivity.
To remedy this, we propose to develop a pixel-based AI to automatically identify body regions in CT and MRI studies.
Previous attempts to tackle this classification problem using supervised and unsupervised deep learning techniques have shown accuracy results ranging from 72% to 92% in limited settings (4,5,6,7,8). Limitations include modality (CT), study design (lack of independent test database, repeat studies, lack of information about image inclusion/exclusion criteria), neural network architecture (legacy neural nets), the size of the database (<1,700 total patients, £100,000 total images), the number of body region classes (£12, no upper extremities), and the extent of clinical protocols covered (mostly thin slice CT acquisitions, no contrast medium). The main study's objective is to demonstrate that the anatomical classifier can reach state-of-the-art accuracy or weighted sensitivity greater than 90% across a large spectrum of body regions, patient demographics, patient comorbidities, and clinical protocols.

Study Design
The performance of AI models was evaluated in a retrospective standalone study using manually defined ground truth data. Three databases were collected for the AI model training and evaluation: training, validation, test (holdout set). The selection of body regions was established with the goal of covering the entire human body. 17 CT (18 MRI) body regions were considered for the classification task: abdomen, breast (MRI only), calf/lower leg, chest, elbow, foot, forearm, hand, head, humerus/arm, knee, neck, pelvis/hip, shoulder, spine cervical, spine thoracic, spine lumbar, thigh/upper leg. Noncontrast scans were collected for all body regions and complemented with MRA and CTA contrast datasets for head, neck, chest, and abdomen.

Data
The data used for training and validation originated from a large cohort of 63,699 de-identified studies (CT: 28,211 patients; MRI: 35,488 patients) from one healthcare system and its affiliates (University of Wisconsin Health). Patients underwent contrast or non-contrast imaging between 1997 and 2020 (median: 2017) and came from a tertiary care hospital as well as a smaller affiliated hospital and several outpatient imaging centers. A well-controlled selection process was used to build the training and validation datasets. In the first phase, the data was randomly selected from this large pool to ensure a balanced distribution of body regions. In the second phase, several iterations of active learning (9, 10) were applied to the large cohort of unlabeled data until the accuracy goal was reached. This step was added to enrich the datasets with more complex anatomical cases.
The test databases were collected from a different source, e.g., United Point Health system (UPH) and its affiliates, so as to address the general issue of model under specification (11). Because of the over-representation of datasets acquired with a GE scanner in the training and validation datasets, the instruction was to focus on collecting datasets from multiple institutions where a GE scanner was not predominantly used. The same attention to collecting a balanced representation of all the body parts was given. The CT and MRI imaging data from 3,003 patients scanned between 2016 and 2020 (median: 2020) was collected and served as a pool of data to build the test databases. These patients came from primary care hospitals, critical care hospitals, and imaging centers. This all-comers study was designed with the intent to be as inclusive as possible and clinically relevant. All randomly selected patients were included in the study irrespective of their demographic (ex: ethnicity) or comorbidities. Due to the under-representation of pediatric cases, test databases were only composed of adult patients. The same inclusive approach was followed for acquisition protocols, orientations (supine, prone, lateral), CT kernels and MRI sequences. The only exclusions were applied to protocols where the anatomical region was faded and challenging to distinguish such as flow protocols and MRI elastography (Supplemental Materials -Inclusion and exclusion criteria). Models were trained on 2D transversal slices. Axial images with an angle up to 45 degrees were included to cover oblique acquisitions. Multiplanar reformat series such as coronal and sagittal series as well as series used to aid planning of the acquisition such as scout, calibration, and quality control series were excluded. Similarly, all post-processed series such as perfusion maps, reformat, 3D reconstruction, secondary capture, CAD, CINE and movies were not considered. Image exclusion criteria were image stored in a format 8 bits or lower, image with multiple channels (RGB or other not grayscale), no pixel data, very limited amount of pixel data (<1,000 pixels), and no compatible codec (anything not JPEG lossless, Raw RLE).
All data were de-identified by partners on site prior to any processing to comply with HIPAA requirements. The deidentification schema followed the one used in the Cancer Imaging Archive initiative (12).

Ground Truth
The ground truth was labeled based on clearly defined anatomical landmarks (Table 1) using a home-grown annotation software (Fig 1). To avoid the effect of inter-reader variability, all the datasets were labeled by one image annotator with a long-time experience of developing CAD solutions (PR). Pediatric patients under 10 years old were reviewed and edited independently by an experienced pediatric radiologist (RAC) but were left out of the final analysis because of insufficient number of samples. To evaluate the accuracy and consistency of the ground truth, an independent review of a random subset of the labeled dataset was conducted by an experienced radiologist (JWP). Consistent with peer review practices

Data Partitions
Repeat studies in the validation and test databases were removed so one study corresponds to one patient. Labeled datasets were organized according to the main body region and sorted according to study size. A 75/25 split between the training and validation patient datasets was performed for each body region. To estimate the size of the test databases, a strict survey study sampling model (14) was used with the assumptions of a model at least 90% accurate and a 95% confidence interval with a 10% relative error. Based on this model it was determined that at least 7,600 images per body region for CT, and at least 3,600 images per body region for MRI were needed.

Model
The classification task is composed of multiple stages that are detailed in Fig 2. As a first step, we used the standard ResNet50V2 model (15) in a multi-class framework. Following the 2D CNN classifier, a few post-processing steps at the series level were applied. First, a rule engine merged the abdomen-chest class to either the abdomen or chest class and classified an entire series as breast if at least 50% of the images in the series were classified as such. Second, a routine was applied to remove MRI series with uncertain results based on the evaluation of decision margin. The goal was to discard series with little anatomical content and severe artifacts. Last, outlier labels not related to the anatomical content of the series were detected and the image predictions smoothed out so a continuity in classified labels could be observed throughout the series.

Training
We enriched our dataset by applying spatial deformations to a random set of images in each training epoch. These transforms include rotation within an angle of ± p/10, translation and shear with a maximum of 10% in image size in both directions, scaling with a maximum of 20% in image size in both directions, and bilinear interpolation. The transformations were applied using built in TensorFlow library functions. We used the transfer learning approach and model weights from pretrained Resnet50V2 model developed for vision benchmark ImageNet dataset. The training hyperparameters are listed in Supplemental Materials -Training Parameters. The loss function used is categorical cross entropy which is well-suited for the multi-class case.

Evaluation
We applied the models to the test datasets and evaluated them by computing the average weighted values of the following performance metrics: sensitivity and specificity. Because of the data distribution, accuracy and aggregated weighted sensitivity are the same so only the latter is reported in the paper. Results were derived by body region, institution, patient demographics, and acquisition parameters (manufacturer, contrast, CT kernel, slice thickness, sequence type).
Performance metrics and their corresponding confidence intervals were determined using the jackknife bootstrap resampling method (16). Random image sampling every 10mm was performed at the series level to reduce the impact of strongly correlated images and provide more realistic statistical results. This is consistent with the 7.5mm sampling approach reported in (17). In order to reduce the inter-series correlation, only one of the series (randomly selected at each sampling iteration) in the subsampled dataset was kept. The correlation and correlation significance between the model's accuracy and each confounding factor was assessed using Cramer's V and Pearson's chi-squared statistical test.

Data
The data consisted of 2,891 CT cases  which represents an error rate of 0.3% per study. There were no cases where the labeling discrepancy would be considered clinically relevant, if for example, two physicians were discussing a patient and using the discrepant labels interchangeably. Tables 2-3. 27 institutions contributed to each CT and MRI Test dataset. For CT, 56% of datasets came from primary care hospitals and 44% from critical access hospitals and imaging centers while for MRI, 55% of the datasets came from primary care hospitals.

Distributions of images and results by confounding factors for the Test sets can be found in
Gender parity was respected for the CT database. A slight over-representation of female gender was noticed for the MRI database (56.1%). The age coverage ranged from 18 years old to +90 years, roughly following the distribution of imaging tests in US Healthcare Systems (18). Compared to the development databases (Supplemental Materials -Distribution Development Databases), the test databases differed in some key areas. For CT, acquisitions mostly originated from Siemens and non-GE scanners (87.6%, +80.1%) with a larger proportion of older adults ³65 years (45.6%, +9.3%), intermediate slice thickness (2mm < slice thickness < 5mm) (54.8%, +46.3%), and non-contrast imaging (76.5%, +9.1%).

Model Performance
An overall body region image-level sensitivity of 92.5% (92.1 -92.8) was achieved for CT and 92.3% (92.0 -95.6) for MRI. The post-processing stages contributed to about 1.1% (CT) to 1.6% (MRI) improvement in classification accuracy.
Classification results by body region are listed in Table 4. Accuracy results were consistent across body regions. Head and breast images have very discernable features, so they tend to be more accurately classified than other body regions such as the neck and extremities. Confidence intervals for sensitivity was lower (upper bound did not include 90%) for the following anatomical regions: CT cervical spine, CT forearm, CT pelvis, MRI cervical spine, MRI forearm and MRI neck.
No formal association was found between classification accuracy and CT institution, CT kernel, MRI contrast. However, statistically superior classification results were noticed in a few instances with Cramer's V correlation ranging from For some of the classes in the test sets, the association between accuracy and factors such as manufacturer and MRI sequence could not be reliably assessed: Hitachi and Canon scanner manufacturers and In and Out of Phase MRI sequences. In spite of these limitations, the evaluation of accuracy results and confidence intervals points to performance robustness across age, manufacturer, CT slice thickness and MRI sequence categories.

DISCUSSION (683 words)
This work demonstrates how a deep learning CNN based classifier can achieve overall state-of-the-art accuracy greater than 90% in identifying body regions in CT and MRI studies while covering the entire human body and a large spectrum of acquisition protocols across separate institutions. This is the first known attempt to a) provide a solution for MRI, and b) offer a clinically relevant solution that covers numerous classes, in particular extremities that are more prone to misclassifications.
There are several reasons why extremities are challenging to classify. First, extremities can be imaged in multiple orientations depending on the patient's clinical condition. This means that in some cases, an axial series may present in the form of an oblique axial or even another MPR view. The training database was built to account for such a diversity of orientations. Second, extremities are subject to fractures, surgical implants and imputation, which can be sources of body region misclassification. Third, images of upper and lower extremities have structural similarities that can make them look similar in some circumstances. This is the case for joints, paired long bones of the calf and forearm, single long bones of the thigh and arm, and knee and elbow. Fourth, the boundaries between body regions for the upper and lower extremities are less distinct than between other body parts.
Although slice thickness and FOV can be informative, the differentiation between a neck and a cervical spine study can be sometimes difficult in particular for soft tissue reconstructions. The clinical impact may be minimal though since both body regions represent the same anatomical region. At the time of field deployment, the user can decide to configure the application so that these two classes are merged.
The study has some limitations. First, the multiclass framework is, by design, not well suited to identify multiple regions in an image. This is a limitation when dealing with whole body studies. Second, the ML models were trained and evaluated on axial images. The same way data from the axial plane are reconstructed to non-axial planes, anatomical predictions determined in the axial plane can be easily ported to coronal and sagittal planes. Third, there are classes in the databases that are under-represented such as Hitachi and Canon scanner manufacturers, CT thick slices ³5 mm, and In and Out of Phase MRI sequences. Additional data is needed to complement the evaluation for these classes.
As stated in a joint paper by HIMSS and SIIM, the implications of our work are multiple (1). An improved method for anatomic labelling of imaging studies has the potential to improve interoperability across healthcare records and systems, address radiology workflow challenges such as labelling discrepancies for studies shared within and between facilities (19), accurate retrieval of anatomically relevant comparison images from image archives and bandwidth related latency and costs associated with data retrieval from cloud-based image archives. Application of our body region labelling could improve display protocols, image synchronization and relevant prior functions in PACS, leading to improved radiologist accuracy and productivity. Beyond image interpretation, it could enable body region dependent AI driven population health initiatives across institutions. When mining the DICOM tags in the test sets for either the "BodyPartExamined" DICOM tag (BP) or "ProcedureType" (PT), the body region information at the study level was only 22.3% (BP) and 42.2% (PT) accurate for CT, and 58.3% (BP) and 47.8% (PT) accurate for MRI. In this cohort, the anatomical AI has the potential to improve the search for anatomically matched cases for more than 50% of the cases.
Automatic identification of body regions in CT and MRI studies in a general population is a challenging task due to the large spectrum of demographics, comorbidities, acquisition protocols, and imaging artifacts. In this context, the anatomical AI demonstrated state-of-the-art performance with an overall image-level classification accuracy greater than 90%, and performance metrics robust across all body regions and confounding factors such as institution, gender, contrast, manufacturer, slice thickness, CT kernel, and MRI sequences. Our next step is to extend the evaluation to other data sources and conduct an observer study to assess the impact on radiologist workflows.   Lower in the scan, thigh images (purple) are also correctly predicted.

Fig. 2:
Body region classification steps. In order to accommodate the model's input, several pre-processing steps were applied at the image level. First, pixel values were clipped to fit the interval [mean -4*std, mean + 4*std], where mean and std correspond to the mean value and standard deviation of pixels in the image. Second, pixel values were normalized using the following transform (pixel value -mean)/(2*std). Third, the gray scale images were converted to RGB images by copying pixel data from first channel to the second and third channels. Last, each image was resized with zero padding to fit the model's required input size of 224 x 224 pixels. Following the 2D CNN image classifier, two rules were applied at the series level to merge the abdomen-chest class to either the abdomen or chest class depending on which body region was predominant. In the absence of chest and abdomen predictions, an abdomen-chest prediction was classified as abdomen. A second rule was classified as breast when at least 50% of the images in the series were classified as such.
This is to eliminate spurious measurements in noisy breast acquisitions. In order to discard series with little anatomical content and severe artifacts, a routine was applied to remove MRI series with uncertain results based on the evaluation of entropy. In the last two stages of post-processing, we first applied a routine at the series level to remove outlier labels. We then applied a moving average filter with a window size of three pixels to smooth out results so a continuity in classified labels could be observed throughout the series.

Stage 1 Stage 3 Stage 4
Removal of series with high uncertainty (MRI only) Stage 2 Table 2: CT image performance metrics by confounding factors. n=number of studies (*series). The p-value for the median chi-square is provided to determine if a significant difference in accuracy is found for each confounding factor.

Category n (%) Sensitivity (95% CI) Specificity (95% CI) p-value Institution
Primary  Table 3: MRI image performance metrics by confounding factors. n=number of studies (*series). The p-value for the median chi-square is provided to determine if a significant difference in accuracy is found for each confounding factor. **124 studies did not have institution information. ***137 series did not have any of the preset sequence tags. Due to the small number of cases, the performance metrics and confidence interval are not reliable for "In and Out of Phase".  Table 4: CT and MRI image classification sensitivity and specificity by body region. n=number of images.

Inclusion and Exclusion Criteria
Many types of MRI sequences were included in the study such as image weighting, spin echo, gradient echo, inversion recovery, MRA, In and Out of Phase, and diffusion imaging ( Table 1).
Were excluded from consideration in the study, imaging sequences used to determine flow velocities such as VIPR (High

Distribution Development Databases
• Table 4: patient and image distribution by body region (CT) • Table 5: patient and image distribution by body region (MRI) • Table 6: study distribution by institution, demographics, and manufacturer (CT) • Table 7: series distribution by scanner parameters and clinical protocol (CT) • Table 8: study distribution by institution, demographics, and manufacturer (MRI) • Table 9: series distribution by scanner parameters and clinical protocol (MRI)

Results Validation Database
• Table 10: sensitivity and specificity by body region (CT) • Table 11: sensitivity and specificity by body region (MRI) • Table 12: sensitivity and specificity by institution, demographics, manufacturer, scanner parameters (CT) •