{'fraction_non_alphanumeric': 0.030113911778962676, 'fraction_numerical': 0.017571497818710615, 'mean_word_length': 5.309250764525994, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Improvement in Validity and Reliability of Airway Assessment Using Ultrasonography by Novice Personnel: A Learning Cohort Study

Purpose To assess improvements in the validity and reliability of novices’ skills in performing ultrasonography for airway assessment. Patients and Methods A learning cohort study was conducted with 20 anesthesiology residents and 10 volunteers in the Department of Anesthesiology, Faculty of Medicine, Prince of Songkla University. The four parameters of airway assessment were soft tissue thickness at the level of 1) hyoid bone (STT-HY), 2) true vocal cords (STT-VC), 3) thyroid isthmus (STT-TI), and 4) suprasternal notch (STT-SN). The magnitude of discrepancies between the residents’ and experienced anesthesiologists’ measurements was evaluated over the sequence of measurements in the 10 volunteers. Results The mean ultrasonic measurements of STT-HY by the experienced anesthesiologists and residents were significantly different (11.09 ± 3.14 mm vs 8.53 ± 3.02 mm, respectively; P = 0.008), whereas measurements of STT-VC, STT-TI, and STT-SN were not (7.18 ± 1.70 vs 7.14 ± 1.93, P = 0.32; 7.81 ± 2.14 vs 7.73 ± 2.19, P = 0.62; and 11.32 ± 3.33 vs 10.30 ± 3.02, P = 0.35, respectively). The mean discrepancy between the residents’ and experienced anesthesiologists’ measurements was close to zero throughout the sequence of measurements of STT-TI and STT-VC. However, the residents’ measurements of STT-HY and STT-SN were considerably lower than those of anesthesiologists. The range of discrepancies between residents and experienced anesthesiologists in each sequential measurement was wide for all measurements, particularly for the measurement values of STT-HY, and the standard deviation of the discrepancies did not decrease over the sequence of measurements. Conclusion Over the sequence of measurements for airway assessment in 10 volunteers by 20 residents in this learning trial, we found no evidence of improvement in measurement accuracy. Discrepancies between the residents’ and anesthesiologists’ measurements and the variability in discrepancy across residents were greatest in the measurement of STT-HY.


Introduction
Complications in airway management are often unanticipated and can lead to disability or death. The most common complication is an unexpected encounter of a difficult airway, posing challenges in achieving adequate intubation and ventilation by anesthesiologists. Thus, accurate airway assessment is necessary for predicting difficulties in intubation and ventilation, as well as preparing the assistants and the equipment needed to secure the airway. The use of a correct technique for airway management can reduce the risk of severe complications. [1][2][3] Airway can be evaluated by physical examination, including determining the Mallampati classification score (sensitivity, 43-49%), interincisor gap (sensitivity, 46%), thyromental distance (sensitivity, 20-25%), and sternothyroid distance (sensitivity, 62%), as well as performing the upper lip bite test (sensitivity, 7-19%) and flexion and extension of the neck (sensitivity, 10-16%). 4,5 However, these modalities have a low sensitivity, which can lead to inaccurate prediction of difficulties in intubation and ventilation.
Ultrasonography is used in patients with trauma or critical illnesses for vascular assessment, diagnosis, regional anesthesia, and airway management. Ultrasonography for airway management can be performed in the pre-intubation phase to evaluate the pathology and measure the trachea width to estimate the tube size, [6][7][8] intubation phase which detect endotracheal in situ faster than lung auscultation or end tidal CO 2 monitoring, 9,10 and extubation phase to measure diaphragm excusion and predict the extubation success. 11 However, the technique of using ultrasonography for airway assessment is new, and only a few experts use ultrasonography for airway assessment in Thailand. Therefore, skill development in novice personnel pertaining to the use of ultrasonography for airway assessment needs to be investigated. We aimed to assess improvements in the validity and reliability of novices' skills in performing ultrasonography for airway assessment.

Materials and Methods
This learning cohort study was conducted after receiving approval from the Human Research Ethics Committee, Faculty of Medicine, Prince of Songkla University (REC 61-420-8-4). Twenty anesthesiology residents with no experience in using ultrasonography for airway assessment were included in this study. Four ultrasonography parameters of airway assessment were analyzed in 10 volunteers, random number generation from all nurse assistants, by the 20 anesthesiology residents from one institute. This analysis was first performed by one of two anesthesiologists with experience in airway ultrasonography. The exclusion criteria for the volunteers were presence of a neck mass, history of radiation therapy or surgery, wound in the head and neck region, and/or infection in the neck region.
After obtaining informed consent from the residents and volunteers, their demographic data were recorded. A Philips Affiniti 50 (Philips, Amsterdam, the Netherlands) ultrasound machine with a linear probe (L12-5) was used in this study ( Figure 1).
The two anesthesiologists who had the experience of airway ultrasound for 3 years performed ultrasonography three times for airway assessment in each volunteer. These values were averaged to obtain the "gold standard" measurement values for each parameter and standardized by interpersonal validity.
Before the residents performed any measurements, the method of performing ultrasonography for airway assessment was explained, including the assessment of the following four parameters: 1) soft tissue thickness at the level of hyoid bone (STT-HY, distance between the skin and the upper border of hyoid bone; Figure 2); 2) soft tissue thickness at the    Figure 5). The method of performing ultrasonography for airway assessment was explained via Supplementary Video before the time of first measurement by each resident. Measurements for each parameter were recorded thrice in millimeters, and the mean was calculated and recorded.  All 20 residents performed ultrasonography for airway assessment in all 10 volunteers, but in a different order. After the first measurements were performed by each resident, one of two experienced anesthesiologists provided feedback to the residents regarding any errors in the technique, though the discrepancy from the "gold standard" value was not disclosed. All subsequent measurements by the residents on the same and subsequent volunteers were performed on their own, without any feedback by any experienced anesthesiologist. Three measurements were performed for each parameter by each resident on the same day in a particular volunteer. The average values were used to calculate the discrepancy from the "gold standard" value for that parameter in that volunteer.

Statistical Analysis
Statistical analyses were performed using RStudio (version 2.13.0; R Foundation for Statistical Computing, Vienna, Austria) and Stata (version 14.2; StataCorp, College Station, Texas, USA). The magnitude of discrepancies between the measurements performed by residents and anesthesiologists was calculated over the sequence of measurements in the 10 volunteers. Change in discrepancies over the sequence of measurements was evaluated graphically and using mixedeffects random-intercept linear regression models, in which the residents were considered as random elements and the interaction between discrepancy and sequence was considered a fixed effect.

Sample Size
The required sample size was estimated to achieve a power of 80% in detecting a significant difference in the size of the standard deviation (SD) of discrepancies between the first and last measurements in the sequence of measurements at an alpha of 0.05. The sample size for the test of the difference between the two variances was used.
The study hypothesis was that the SD of performed measurements by residents at the tenth time in the sequence would be less than or equal to 0.7 times the SD at the first time in the sequence.
n ¼ 1:96 þ 0:84 Consequently, a study population of 20 residents was estimated to provide an adequate sample size.

Results
The mean age of the residents was 27.4 ± 2.1 (mean ± SD) years. Of the 20 residents, 4 were male, and 8 were first-year, 6 were second-year, and 6 were third-year residents ( Table 1).

DovePress
The mean discrepancy between measurements performed by residents and experienced anesthesiologists was close to zero throughout the sequence of measurements of STT-TI and STT-VC. However, the measurement values of STT-HY and STT-SN were considerably lower when performed by residents than when performed by experienced anesthesiologists throughout the sequence of measurements in the 10 volunteers. The range of discrepancies between measurements performed by residents and experienced anesthesiologists for each sequential measurement was wide for all measurements, particularly for the measurement of STT-HY, and the SD of discrepancies did not decrease over the sequence of measurements ( Figure 6). Regression models including sequences revealed no significant decrease in the discrepancy in any parameter over the sequence of measurements.

Discussion
Accurate airway assessment is essential to predict difficulties in intubation and ventilation. We evaluated the sensitivity of parameters used in this study and compared it with that of parameters such as Mallampati classification (sensitivity,   43-49%), interincisor gap (sensitivity, 46%), thyromental distance (sensitivity, 20-25%), sternomental distance (sensitivity, 62%), upper lip bite test (sensitivity, 7-19%), and flexion and extension of neck (sensitivity, 10-16%). 4,5 The sensitivity of these parameters is low. Therefore, we need tools with a higher sensitivity for promptly securing the airway when a difficult airway is predicted. Ultrasonography has played a key role in many investigations and procedures. Airway ultrasonography is noninvasive, easy to perform, and useful in preparing for the management of difficult airways. Ultrasonography for airway management can be performed in the pre-intubation phase (airway assessment and prediction of tube size), 6-8 intubation phase (ultrasound-guided intubation and confirmation of tube placement), 9,10 and extubation phase (estimation of diaphragmatic function and examination of lung pathology). 11 In this study, we focused on performing ultrasonography for airway assessment in the pre-intubation phase.
We hypothesized that performing airway ultrasonography in at least 10 cases would improve the physician's skill because the technique is not difficult. We designed a study to compare the SD of measurement values for each parameter determined by each resident with those determined by experienced anesthesiologists. Computed tomography (CT) is the gold standard for airway imaging. 12 Sustic et al observed a strong correlation between tracheal width and left main bronchus width with ultrasonography and not CT. 13 However, Prasad et al 14 found that the parameters of hyomental distance, depth of the epiglottis below the hyoid, thyrohyoid distance, and depth of the arytenoid cartilage did not differ significantly between sonographic and CT measurements. Healthy volunteers exposed to radiation while undergoing CT of the head and neck region are at an increased risk of cataract and malignant tumor development in radiosensitive organs such as the salivary gland, thyroid gland, and retina. 12

DovePress
It would have been unethical to expose volunteers to unnecessary radiation for this study. Thus, we designed this study to compare the learning curve of novices with that of experienced anesthesiologists to avoid the possible adverse effects of radiation exposure by CT.
Although CT is the first-choice imaging modality for the larynx and hypopharynx, the values of some parameters measured by CT are significantly different from those measured by sonography, such as the distance to the posterior surface of the tongue, thickness of the submental region, depth of the epiglottis from the skin, and fat pad thickness at the level of the thyroid cartilage. 14 Over the sequence of measurements for airway assessment using ultrasonography in 10 volunteers by 20 residents in this learning trial, we did not find any evidence of increasing validity or reliability compared to that of experienced anesthesiologists. A possible explanation could be that the residents were not given adequate feedback after each measurement. No feedback was provided regarding the magnitude of the discrepancy in measurements. Experienced anesthesiologists provided some feedback about the technique for identifying the structure of the airway for the first measurement on the first day, but not for subsequent measurements. Consequently, the residents never understood whether their measured values were more or less than those of experienced anesthesiologists. Another explanation could be that 10 cases were not adequate to observe the improvement in validity or reliability.
STT-HY measurements can accurately indicate the possible difficulties in laryngoscopy (1.69 cm, 95% confidence interval [CI] = 1.19-2.19 in the difficult laryngoscopy group vs 1.37 cm, 95% CI = 1.27-1.46 in the easy laryngoscopy group). 15 STT-HY measurements can also indicate the possible difficulties in mask ventilation (0.88 ± 0.3 cm for easy mask ventilation vs 1.4 ± 0.019 cm for difficult mask ventilation). 16 STT-VC > 23 mm has been associated with difficulties in intubation (sensitivity, 85.7%; specificity, 57%; and area under the curve, 0.73). 7 STT-TI ranged from 0.78 ± 0.35 cm to 1.08 ± 0.33 cm but could not accurately predict difficulties in mask ventilation and laryngoscopy. 16 STT-SN has been used to verify the placement of the tube at the correct depth after endotracheal intubation. 17 STT-VC, STT-TI, and STT-SN were found to be simple measurements for novices. The measurement values recorded by residents were similar to those recorded by experienced anesthesiologists.
The mean and SD of measurement values for each parameter determined by the residents over the sequence of measurements, as compared with those determined by experienced anesthesiologists, showed that the range of discrepancies was wide and did not change over the sequence. The residents' measurements of STT-HY were significantly lower than those of experienced anesthesiologists. However, the values of STT-VC, STT-TI, and STT-SN were not significantly different over the sequence between the measurements recorded by residents and experienced anesthesiologists. This result can be explained by the compression due to the ultrasound probe. In volunteers with a greater thickness of soft tissue, the ultrasound probe could have led to greater compression. As the residents had no experience in performing ultrasonography for airway assessment, they may have used the ultrasound probe more aggressively than experienced anesthesiologists. This study has some limitations. First, the residents were not given adequate feedback. If they had received qualitative and adequate feedback for each measurement, the discrepancy in the measurement values between residents and experienced anesthesiologists may have improved in each sequence. Second, the small number of volunteers did not permit any analysis of volunteer characteristics that may have affected the accuracy of the residents' measurements. Third, the mean of measurements performed by experienced anesthesiologists using ultrasonography was considered the gold standard. Ultrasonography is an operator-dependent procedure; thus, values measured by anesthesiologists may have been inaccurate. However, both anesthesiologists were validated in the use of ultrasonography for airway assessment using the parameters used in this study. In future studies, values obtained by other imaging modalities, such as CT and magnetic resonance imaging, may serve as a more suitable gold standard.

Conclusion
Over the sequence of airway assessment in 10 volunteers by 20 residents in this learning trial, we did not find any improvement in measurement accuracy. Discrepancies between the residents' and anesthesiologists' measurements and the variability in discrepancy across residents were greatest in the measurements of STT-HY.