{'fraction_non_alphanumeric': 0.03104726598702502, 'fraction_numerical': 0.014249304911955514, 'mean_word_length': 4.852542372881356, 'pattern_counts': {'":': 0, '<': 1, '<?xml version=': 0, '>': 1, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Auxiliary diagnosis of developmental dysplasia of the hip by automated detection of Sharp's angle on standardized anteroposterior pelvic radiographs

Supplemental Digital Content is available in the text


Introduction
Developmental dysplasia of the hip (DDH) is a common disease, with an incidence of about 1 per 1000 and a high disability rate. [1] For a better prognosis, early diagnosis, and treatment of DDH is important, but at the early stage there are often mild or no symptoms. Screening and diagnostic methods do exist, but the shortage of professional orthopedic surgeons in grassroots areas still makes early diagnosis of DDH difficult. Thus, a simple and cost-effective tool to help quickly diagnose DDH from a large number of pelvic anterior images, and reduce the rate of misdiagnosis, is urgently needed.
The most common method for diagnosing DDH uses x-ray images. DDH is characterized by a shallow, steep, and straight acetabular roof. In addition, Sharp's angle is widened, that is, the angle between the lower edge of the pelvic teardrop and the line connecting the lower edge of the teardrop and the outer edge of the acetabulum. The degree of Sharp's angle reflects the acetabular development and the coverage of the acetabulum on the femoral head, and can be used to diagnose and predict DDH progression. Thus, for patients with DDH the determination of Sharp's angle is essential for making clinical decisions. [2,3] In adults and older children with closed Y-shaped triradiate cartilage, Sharp's angle normally ranges from 33°to 38°. Hip dysplasia is suggested when Sharp's angle is greater than 47°. [4] Presently, Sharp's angles are manually drawn and measured on pelvic x-ray images by a professional orthopedic surgeon. However, so many pelvic x-ray images are generated daily in the hospital that it is difficult for orthopedic surgeons to remain current. Furthermore, doctors in some grassroots areas are insufficiently trained to diagnose DDH. Thus, a convenient and efficient method to measure Sharp's angle is urgently needed.
In recent years, the application of artificial intelligence (AI) has led to advancements in speech and image recognition and text understanding. [5] There have also been many achievements in medical imaging research, [6] notably in radiology, [7] dermatology, [8] and ophthalmology. [9][10][11][12] The mainstream models in computer vision have been screened and compared, including a regional convolutional neural network (R-CNN) series, [13,14] the YOLO (You Only Look Once) series, [15,16] and SSD (Single Shot Detection). [17] In this current study, Mask R-CNN was employed [18] to locate 4 key points on x-ray images that correspond to Sharp's angle, and these points were utilized to calculate Sharp's angle automatically.
The validity of this proposed model was evaluated by comparing the automated measurements of Sharp's angle with the manual measurements taken by well-trained orthopedic surgeons.

Methods
This study was approved by the Ethics Review Board of Second Hospital of Jilin University (No. 223 2018). A requirement for patients' informed consent was waived. Patients' data was retrospectively collected and personal information was removed to ensure anonymity. A deep learning artificial neural network to measure Sharp's angle automatically was proposed and developed step by step (Fig. 1).

Datasets
Standardized anteroposterior pelvic radiographs were collected from patients who visited Second Hospital of Jilin University between August 2009 and May 2018. The images were downloaded from the picture archiving and communication system (PACS) of the hospital, intended for modeling and analysis only.
Among the downloaded images, well-exposed standardized anteroposterior pelvic radiographs of patients aged 12 to 100 years were selected by 2 orthopedic professors and 2 radiology professors. Images were excluded from this study if the patient was diagnosed with any of the following: pelvic rotation or tilt, according to the criteria of Tönnis [19] ; postoperative hip replacement or acetabular fracture; diseases including severe hip osteoarthritis or tuberculosis; tumors causing severe changes in hip morphology; severe hyperplasia of the upper edge of the acetabulum; or images with blurred teardrops.
Eventually, 12,528 x-ray images of patients who underwent Digital Radiography exams were initially included in this study, based on the inclusion and exclusion criteria. Of these, 954 were removed during the labeling process due to poor image quality, 101 were randomly selected as test data (not used for deep neural network training), and the remaining 11,473 constituted a training and validation dataset at a ratio of ∼8:2 (i.e., 9248 and 2225 images, respectively, training and validation).

Labeling the pelvic x-ray images
The labeling tool LabelMe was utilized to mark the 4 key points (the lower edge of the teardrop and the outer edge of the acetabulum on both sides) that comprise Sharp's angle on standardized pelvic x-ray images (Fig. 2). Each labeled x-ray image was examined twice before inclusion in the training dataset. Each annotated image with marked corresponding key point coordinates was stored in a separate * .json file.

Preprocessing training images
The x-ray images included in this study were generated using different equipment, so each was first scaled in x Â y pixels (x = 1024) while maintaining the original image's aspect ratio. For model training, the images were made to fill a 1024 Â 1024 pixel square. The pixel mean was deducted from the RGB (red, green, blue) channels before training. To apply to the modified Mask R-CNN model, the * .json file with the annotated data was converted into an MS COCO style annotation file by our own Python-based utility software. A custom key category and a square generated with the key points returned the vertex coordinates of the border.

Modifications of the Mask R-CNN model
The standard Mask R-CNN model was selected [18] and fine adjustments were made based on it. ResNet101 + FPN (feature pyramid network) was still used as the backbone of the network, which was consistent with the standard model in the literature. According to suggestions in the literature, [18] the header network portion was fine adjusted. In addition to the existing classification/border prediction/mask prediction branch, a parallel branch was added for key point detection (Fig. 3). When dealing with key point detection, each key point was treated as a one-hot binary mask whose dimensions were tested and optimized to a value of 56 Â 56 pixels. For model training, the irregular quadrilateral (see Fig. 3) was first treated as the only category (excluding the background category) to predict the position of the corresponding 4 key points. However, the results showed that the performance using the test data was not ideal. Our Python-based utility was then modified to label the area near each of the 4 key points as a separate category and automatically generate a small square centered on the original key point. This gave us 5 categories, which included the original irregular quadrilateral (see Fig. 3) and the 4 new categories described above.
Each of the 4 vertices of the small red squares in Fig. 3 was considered a new key point, so 4 additional categories provided the corresponding additional 4 Â 4 key points. The predictions on the test set data were then the coordinates of the center point calculated from the vertices of each small square border. With this new training strategy, the performance on the test data set was significantly better than the previous one. We suspected that if the number of key points was insufficient, there might be an overfit of the model capacity. The loss function in this study was basically consistent with the loss function described in a previous research, [18] and we added the loss function for key point detection. For each key point, the cross-entropy loss of its Softmax was minimized during training.

Training and inference
Four graphics processing units (GPUs) were used in a mini-batch and 2 images per GPU resulted in an effective batch size of eight. A region of interest was considered positive if the intersectionover-union was at least 0.7; otherwise it was negative. The ratio was 1:2 for positive-to-negative samples of each image. Stochastic gradient descent was used with an initial learning rate of 0.002, a momentum of 0.9, and a weight decay of 0.0001. First, the rest of the network was fixed and the head network iteratively learned 30 epochs. At this point, the validation loss started to show a trend of flattening and small increase, which indicated a possible overfitting to the network. (See details in Fig. 4). The initial learning rate was divided by 10 and all network layers were opened to learn another 50 epochs to achieve the last result. As shown in Fig. 4, although the training loss continued to decrease a bit, the validation loss started to increase after 80 epochs (30 + 50).
The X-ray images in the training dataset were utilized for training and validation at a ratio of ∼8:2. In the test phase, the image was scaled to x Â y pixels (x = 1024) while maintaining the original image's aspect ratio.

Code
The open source project matterport/Mask_RCNN (https:// github.com/matterport/Mask_RCNN) from GitHub was modified to fit the training objects and goals of the present study. A pre-trained weight file (mask_rcnn_coco.h5) based on the MS COCO dataset was used to start the actual training.

Measurement and diagnosis
The proposed AI model was applied to the 101 anteroposterior pelvic radiographs in the test dataset. The lower edge of the teardrop and the outer edge of the acetabulum were then detected. The positions of the 4 key points were predicted and Sharp's angle was drawn and automatically calculated via our Python-based utility (See details in Fig. 2). Accordingly, Sharp's angles between 33°and 38°were considered normal. Sharp's angles less than 32°a re uncommon and probably of no clinical significance, whereas angles from 39°to 42°are the upper limit of normal. Sharp's angles over 47°are characteristic of DDH. The prognosis for hip joints with Sharp angles between 42°and 47°is under investigation and requires dynamic observation [4] according to the diagnostic criteria.
The Sharp's angles of the 101 test orthotopic anterior pelvic Xray images were also measured traditionally by 3 rigorously trained attending orthopedic surgeons using the PACS tools. A DDH diagnosis was based on the measured angles. To evaluate efficacy, the measurements of Sharp angles of the acetabulum performed by the AI model and surgeons were compared with the clinical DDH diagnosis (based, as standard, on medical history, symptoms, signs, x-ray films, and computed tomography [CT] images).

Statistical analysis
SPSS 21.0 statistical software was used to analyze and organize the data. The count data are represented by the number of cases and percentage rate, i.e., n (%). Comparisons between groups were performed by Chi-Squared test (x 2 ). Measurement data are described as mean ± standard deviation. According to the Kolmogorov-Smirnov normality test, P > .05 indicated that the data conformed to normal distribution.
Comparisons between groups were analyzed by variance Ftest. Two groups were compared using the least significance difference test and Student t test. The consistency of the indicator diagnosis results was tested by the kappa statistic. Statistical significance was set at P < .05.

Results
The Sharp's angles of the 101 test anteroposterior pelvic radiographs, measured by different approaches, were compared    (Table 4, Supplementary Table 3, http://links.lww. com/MD/D507). The kappa test indicated that the AI method and surgeons all shared a moderate diagnostic consistency with the final diagnosis results, with no significant difference among them. Thus, the consistency of Sharp's angle-based evaluation of the acetabulum by the AI method suggests that it could have an important role in assisting the diagnosis of DDH.

Discussion
The deep neural network model is currently used in image research for image classification, object detection, and semantic segmentation. Researchers should pragmatically select what is most appropriate for the clinical application. For the present study, we considered that a classification judgment was required with regard to DDH diagnosis. However, surgeons do not intuitively accept a classification judgment achieved via the black box approach of an artificial neural network. [20] Therefore, we determined 4 sites that were essential for measuring Sharp's angle. The present model accurately locates the 4 key points, which are on both sides of the outer edge of the acetabulum and the lower edge of the teardrop. Sharp's angles were then drawn from these points. Thus, the AI model and the traditional manual method used by surgeons share the same measurement procedures for Sharp's angle. Mask R-CNN was mainly used for semantic segmentation and to site the key points on the human body. For the first time, key point positioning was applied to medical x-ray images to calculate meaningful medical   When fine-tuning the proposed AI model, we found that the initial performance was not ideal. This may be because too few key points caused an overfit of the model capacity. When we increased the number of key points and categories appropriately, the performance of the model significantly improved. It was also determined that the addition of a parallel branch for key point detection to the classification/border prediction/mask prediction branch of the standard Mask R-CNN did not impose a significant additional burden on the training of the model. Doctors in China have very heavy workloads. Their stressful working conditions not only threaten their physical and mental health, but negatively influence the quality and efficiency of medical services and patient safety. [21] The AI model proposed in the present study was able to measure Sharp's angle 74 times faster than the physicians could (Fig. 5). This suggests that the AI method could relieve the heavy daily workloads of orthopedic surgeons by improving their work efficiency.
In addition, the diagnostic accuracy of DDH based on the measured Sharp's angle by the AI model was comparable to that of the surgeons (Tables 2 and 3). The AI model could be applied in primary care hospitals, which provide 80% of the clinical care services in most countries, [22] even while the number of doctors and the available technology remain insufficient. In China for example, only 17.6% of the medical undergraduates and above become doctors in township hospitals. [23] Application of the AI model in grassroots medical facilities will help differentiate those patients with an abnormal Sharp's angle, and could thus improve the accuracy and efficiency of diagnosis and treatment. As application of the AI model improves the efficiency of doctors' daily work, the egalitarianism of medical care will improve, and the shortage of doctors and technology may be compensated to some extent.
In clinical practice, in almost 40% of young and middle-aged patients hip pain is caused by dysplasia. Yet, dysplasia due to minor poor acetabular development is often not serious. As observed on pelvis x-ray images, Sharp's angle is only slightly wider than normal, [24,25] and these patients are very susceptible to misdiagnosis. Our present results showed that the diagnostic accuracy of DDH based Sharp angle as measured by the AI model was equivalent to that of the surgeons. In addition, it took much less time for the AI model to perform the measurement, making batch screening possible. By aiding diagnosis of DDH at the early stage, this AI method can contribute to better clinical decisions and treatment strategies, and improve the prognosis before cartilage degeneration occurs.
The overall accuracy of the automated auxiliary detection method for DDH is 76.73%. A final clinical diagnosis requires multiple indicators. This method can contribute to an accurate diagnosis, but there are many challenges to its ultimate application in the clinic. [20] In the future, a CNN model which measures the center-edge angle and the acetabular index will be constructed with the existing data. CT and magnetic resonance imaging (MRI) data can be used to train the CNN model; CT images can better exhibit the acetabular dysplasia [26,27] and MRI depict joint changes in the cartilage. [28] The multi-modality data (x-ray, CT, and MRI data) of the trained CNN model will more accurately assist the diagnosis of DDH.

Conclusion
This study proposed a new method for auxiliary diagnosis of DDH. The method utilized deep neural network AI to detect key points automatically on pelvic x-ray images at the lower edge of the teardrop and the outer edge of the acetabulum. These were utilized for automated calculation of the acetabular Sharp's angle. We improved the Mask R-CNN model as needed for detection of the key points on medical images. The sensitivity, specificity, and accuracy of this method for auxiliary diagnosis of DDH are similar to that of surgeons, while it is much more time efficient.