{'fraction_non_alphanumeric': 0.02438559725662031, 'fraction_numerical': 0.012335682987235664, 'mean_word_length': 5.2602862254025045, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 3, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Automated Diabetes Case Identification Using Electronic Health Record Data at a Tertiary Care Facility

Objective To develop and validate a phenotyping algorithm for the identification of patients with type 1 and type 2 diabetes mellitus (DM) preoperatively using routinely available clinical data from electronic health records. Patients and Methods We used first-order logic rules (if-then-else rules) to imply the presence or absence of DM types 1 and 2. The “if” clause of each rule is a conjunction of logical and, or predicates that provides evidence toward or against the presence of DM. The rule includes International Classification of Diseases, Ninth Revision, Clinical Modification diagnostic codes, outpatient prescription information, laboratory values, and positive annotation of DM in patients’ clinical notes. This study was conducted from March 2, 2015, through February 10, 2016. The performance of our rule-based approach and similar approaches proposed by other institutions was evaluated with a reference standard created by an expert reviewer and implemented for routine clinical care at an academic medical center. Results A total of 4208 surgical patients (mean age, 52 years; males, 48%) were analyzed to develop the phenotyping algorithm. Expert review identified 685 patients (16.28% of the full cohort) as having DM. Our proposed method identified 684 patients (16.25%) as having DM. The algorithm performed well—99.70% sensitivity, 99.97% specificity—and compared favorably with previous approaches. Conclusion Among patients undergoing surgery, determination of DM can be made with high accuracy using simple, computationally efficient rules. Knowledge of patients’ DM status before surgery may alter physicians’ care plan and reduce postsurgical complications. Nevertheless, future efforts are necessary to determine the effect of first-order logic rules on clinical processes and patient outcomes.

D iabetes mellitus (DM) is a metabolic disease resulting from abnormal insulin secretion or insulin resistance, or both. Whether DM is type 1 (type 1 diabetes mellitus [T1DM]) or 2 (type 2 diabetes mellitus [T2DM]), it is the leading cause of microvascular complications, myocardial infarction, stroke, congestive heart failure, and end-stage renal disease that often results in premature death or disability. 1 Patients with DM also undergo surgical procedures at a higher rate than do patients without DM. 2 Metabolic stresses experienced in the perioperative encounter can alter glycemic homeostasis of patients with DM, which then may result in higher rates of perioperative hyperglycemia, postoperative sepsis, impaired wound healing, and ischemia. 3 In addition, previous studies have shown that postoperative complications such as stroke, urinary tract infection, postoperative hemorrhage, transfusions, wound infection, and even death are more common among patients with uncontrolled DM. 4,5 Thus, it is imperative to identify patients who have DM before their care episode and initiate appropriate care protocols to optimize their glycemic levels. 3,6,7 In light of the importance of identifying patients with DM before they receive health care, our objective was to develop a highly sensitive and specific automated electronic phenotyping algorithm for such identification. The algorithm would use available data in the electronic health record (EHR) to drive coordinated clinical processes of care. For instance, results from this algorithm would be used by physicians to facilitate 1 or more of the following steps: (1) schedule a regular glucose check, (2) initiate a protocol-driven DM care pathway, (3) initiate a consult with the diabetes service, or (4) ensure insulin availability in the preoperative care environment. Thus, in addition to high classification performance and computational efficiency, the interpretability of the classification process by care providers was prioritized during the development of this DM classifier.

METHODS
In 2012, planning was initiated for the development of an automated DM identification algorithm at Mayo Clinic, a tertiary care academic medical center with 110 operating rooms and more than 1200 beds in Rochester, Minnesota. Mayo Clinic's Integrated EHR complies with all the definitions of a comprehensive EHR and at its core provides physicians with the ability to capture patients' demographic characteristics and medical history, which includes documenting patients' medical issues in a free-text format in patients' notes. 8 The institution performs approximately 4000 surgical procedures a month, with an estimated 15% to 17% of surgical patients having DM. The surgical practice of Mayo Clinic began a project to standardize the care of patients with DM across the surgical episode. During this process, the practice determined that it would be more efficient, effective, and reliable if the DM diagnosis was captured at a central location in the EHR before the patient arrived on the day of surgery. Clinicians agreed that the data elements needed to determine whether a patient has DM have frequently existed in the EHR, but the lack of clear identification of a DM diagnosis in the EHR was a barrier to implementation.

Data Sources
Previous studies have suggested that the most accessible and reliable sources for DM cohort identification were (1) International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) diagnoses codes (Supplemental Appendix 1, available online at http://www.mcpiqojournal.org); (2) laboratory test values; and (3) patient medication data, in varying combinations and thresholds. 9,10 In addition to these 3 primary domains, several studies have indicated that natural language processing (NLP) can considerably increase accuracy and precision during identification of health issues documented in clinical notes. 11,12  As patients treated with metformin for indications of DM may have HbA 1c values that are within normal limits, a decision was made by our endocrinology colleagues to classify patients with any value of HbA 1c and a prescription for metformin in their EHR as having DM.

Algorithm Design
We used first-order logic (if-then-else) rules to imply the presence or absence of T1DM and T2DM. This rule-based classification model had a series of logical statements using logical and operators and logical or operators. Beyond modeling actual human decision making with logical statements, the approach based on the if-then-else rule strikes a balance between accuracy and interpretability for general classification problems. 13 An overview of the classification model is provided in Figure 1. With this method, our rule classified a patient as having DM when the patient's EHR contained the following: 1. One or more outpatient diabetes-related ICD-9-CM diagnosis codes or 2. At least 1 hypoglycemic medication reported during outpatient medication reconciliation or 3. A combination of metformin use and a laboratory value exceeding the maximum threshold value  In light of such limitations, as well as our noted desire to use the results from the developed algorithm for DM determination before the surgical encounter, we were motivated to develop a rule set that could identify accurately the presence of DM in the preoperative environment. This work was performed in collaboration with our endocrinology colleagues to identify a patient who has either T1DM or T2DM at 1 or 2 days before the scheduled surgery date. adopted a stratified sampling approach based on Begg and Greenes method, whereas Newton et al 27 adopted an iterative approach where information obtained at each step is used to fine-tune and improve the final phenotype algorithm method. In our study, we followed the iterative approach suggested by Newton et al and evaluated the performance of the proposed algorithm for patients scheduled for surgery during August 2014. Data related to these patients were extracted from the existing legacy Mayo Clinic Unified Data Platform that stores structured, unstructured, and other patient careerelated data elements from various sources that support research and quality improvements. 28 The ascertainment of true DM incidence as the reference standard was created by comparing results from the proposed method with those of the existing process, which is a manual review of the EHR for a DM diagnosis by bedside nurses. When the bedside nurse identifies DM, its documentation is noted in the preoperative evaluation patient flow sheet. This nurse preoperative evaluation patient flow sheet document serves as a surgery intake tool, nursing communication tool, and assessment tool. The nurse documents whether the patient has DM or a history of DM in the flow sheet. This is based on patient response, surgical listing information, the patient's current medication regimen, and a clinical notes review. The nurse synthesizes this information and documents in the EHR if the nurse confirms that the patient has DM. To develop this reference standard, we first considered concordant cases with agreement between the results from the proposed method and the current manual process (eg, both suggest the presence of DM or neither suggests DM). Among such concordant cases, a randomized sample of 100 patients each (ie, a total of 200 concordant cases) was screened by an expert reviewer, who has 35 years of clinical nursing experience, including patient classification, and 16 years of experience in chart abstraction. This review was to confirm whether the patients indeed did or did not have DM (N¼200). Second, we considered all discordant cases (ie, a total of 231 discordant cases) where the proposed method disagreed with the findings from the current manual process. All discordant pairs were screened manually by the independent reviewer to determine whether the patients had DM. Thus, a total of 431 cases were reviewed. Final determination of the presence or absence of DM involved manual review of a patient's EHR by a research nurse specifically trained in the extraction of medical conditions of interest from the EHR. The decision to manually review a random sample of concordant cases was determined a priori and was arbitrarily based on the collective perception of the research team.

Algorithm Comparison
After developing a reference standard and validating the proposed phenotyping algorithm, we compared the performance of this proposed algorithm with the performance of existing comparator algorithms, as well as current manual imputation of a DM diagnosis by bedside nurses (Table 1).

Statistical Analyses
Measures of diagnostic accuracy (sensitivity, specificity, positive predictive value, and negative predictive value) and 95% CIs were computed for each case-definition method, and the best method that identified patients with DM most accurately was determined on the basis of McNemar test for sensitivities and specificities. 29 Figure 2. A single false-positive finding was noted with the keyword searches because of documentation noting "denis [sic] diabetes" instead of "denies diabetes." Of the other 3524 patients identified as not having DM, 2 patients were incorrectly classified as not having DM by the proposed algorithm (falsenegative results). This misclassification was due to a lack of access to source systems that contained DM-related information for these 2 patients. The details are graphically represented using the classification tree diagram of Figure 2. Of note, the rules implemented in the proposed DM phenotyping algorithm are not mutually exclusive. Nineteen patients were identified as having DM because of the coexistence of metformin and an abnormal DM laboratory value and also met the criteria for DM on the basis of the presence of a DM-related keyword in their clinical notes. Similarly, the cases identified because of overlapping inclusion criteria are depicted in Figure 3.
Compared with the 684 patients identified by our proposed approach, the number of DM cases identified by the comparator methods varied considerably: eMERGE, 377 cases; CCW, 460 cases; DDC, 815 cases; SUPREME-DM, 543 cases; HbA 1c of NYC, 350 cases, and the Harvard Medical School method, 1043 cases. The current manual approach identified 626 patients as having DM (Table 1).
Our proposed method resulted in a sensitivity of 0.9971 (95% CI, 0.9895-0.9996) and a specificity of 0.9997 (95% CI, 0.9984-1.00). The sensitivity of the present phenotyping algorithm exceeded the sensitivity of all other existing methods considered in our study ( However, statistically significant differences in specificities were observed between our approach and the approaches recommended by DDC, SUPREME-DM, HbA 1c of NYC, and Harvard Medical School and the manual approach.

DISCUSSION
With the rapid adoption of EHRs and the development of innovative data extraction strategies, many institutions have initiated or have completed the development of automated computable case-definition methods.
Our results indicate that most of these preexisting methods achieved a high sensitivity of patient identity. Yet, the degradation in the sensitivity of these preexisting methods compared with the current proposed method has many causes. For instance, SUPREME-DM requires that a patient have at least 2 abnormal laboratory values or outpatient ICD-9-CM diagnosis codes not more than 2 years apart ( Table 3). The DDC requires 2 abnormal laboratory values not more than 1 year apart. This requirement excludes patients with a single abnormal laboratory value or ICD-9-CM diagnosis codes. Such data result in increased false negatives and reduced sensitivity. CCW and HbA 1c of NYC are single-element case-definition methods. The CCW requires only patient DM-related ICD-9-CM codes, and HbA 1c of NYC requires only abnormal HbA 1c values and does not identify patients with other indicators, such as DM-related medications or DM-related ICD-9-CM diagnostic codes and resulting in large false negatives and lower sensitivity.  The eMERGE DM algorithm requires at least 2 different DM-related data elements to classify patients as having DM and ignores patients with a single instance of either DM-related ICD-9-CM code or medication or metformin and abnormal values. The algorithm does not categorize these patients as having DM, thus lowering its sensitivity. In addition, neither method considered exploring the DM cases mentioned by the physicians in free-text patient notes data. This point further contributed to lower sensitivity among these different comparator methods. Similar concerns exist when considering the varied specificities of the alternative algorithms.

Strengths and Limitations
A major strength of this investigation is its inclusion of information from multiple domains relevant to DM. This included ICD-9-CM diagnostic codes, medication information, laboratory findings, and keyword searches mining unstructured text contained within clinical notes.
Several investigators have emphasized the importance of incorporating information buried within a patient's clinical notes when attempting to identify patients with specific disease conditions. [34][35][36] Sohn et al 37 noted that rule-based algorithms performed well when NLP-extracted attributes from clinical text were incorporated into the algorithm design. Peissig et al 38 studied cataract cases and asserted that incorporating concepts from clinical notes increased the case identification by a factor of 3 compared with using a sole mode-structured data approach. We used a combination of complex Structured Query Language embedded with Boolean logic ("and" "or") to detect either negation or positive annotation indicative of DM in patient notes (Supplemental Appendix 3, available online at http://www.mcpiqojournal.org). Within such a construct, we were able to identify several dominant patterns and develop a set of inclusion and exclusion criteria that indicated the presence or absence of DM. In addition to identifying various affirmation statuses, negation had a notable role in identifying patients who did not have DM and in improving the accuracy of our algorithm. Unlike confirmatory avowals, negations are commonly indicated by such terms as no, NA, and unknown, so initially we started with these 3 basic terms and iteratively refined our algorithm by adding more negation clauses, such as brothers, one of whom has diabetes type, insignificant for diabetes mellitus, doesn't have a history of diabetes, diabetes screen: fasting glucose ¼ NA, and diabetes screen: unknown if ever checked, and many others during the iterative validation process.
This combination attributed 79 additional cases identified by our algorithm, resulting in performance superior to that of other approaches that did not include unstructured data. Thus, it follows that such a text-parsing NLP algorithm that uses simple keywords to negate or validate the presence of DM will be an EHR systemeagnostic and highly portable that can be implemented in other EHR environments.
Inclusion of metformin as an additional requirement for patients with abnormal laboratory values is another distinctive aspect of our algorithm that may have resulted in higher specificity than with other available phenotyping methods. Our method also did not mandate multiple observations per patient at various time intervals. This proposed method has the utility of identifying patients before their surgical encounter rather than after it. Finally, implementation of the manual review for final determination of DM by a trained study coordinator enhanced our confidence in accuracy regarding the presence or absence of the condition of interest.
Several study limitations merit discussion. First, this investigation represents a single institution's experience in designing, testing, and implementing a DM case-definition algorithm compared with multi-institution casedefinition methods, such as eMERGE, SUPREME-DM, and Harvard Medical School. Further validation and confirmation of the present findings in alternate health care settings across various health care institutions are necessary to understand the generalizability of our proposed method. Many health care institutions will not have access to some of the data elements proposed by our casedefinition strategydmost notably, the ability to query unstructured data for key terms of interest. Although we were able to validate the results from our algorithm using an independent nurse reviewer, we were unable to completely mask the details to the reviewer during the validation because of the iterative nature of algorithm development.
In addition to these concerns, we acknowledge the potential for bias due to the use of a reference standard and not a criterion standard developed by an independent reviewer to whom results were blinded completely while evaluating the accuracy of our proposed method in our study. However, the CIs around our sensitivity and specificity estimates support the stability of our results to our results and mitigate concerns related to the use of such reference standards during the validation process. Currently, a more formal validation of the performance of our proposed method in a clinical environment at our institution is under way.
Although the use of International Classification of Diseases, Tenth Edition codes is the current standard, this study used ICD-9-CM codes. This course was taken because the standard at the time of study onset was ICD-9-CM. Moreover, the comparator algorithms (DDC, CCW, eMERGE, Harvard Medical College, Supreme-DM) were also developed using ICD-9-CM codes. Nonetheless, we acknowledge the use of ICD-9-CM codes in our study as a limitation, and we have implemented a strategy to map the ICD-9-CM codes associated with this study to their corresponding International Classification of Diseases, Tenth Edition codes.

CONCLUSION
We developed an efficient and accurate DM phenotypic algorithm that outperformed other available approaches. This algorithm facilitates the early recognition of DM, thereby permitting the implementation of improved workflows for optimal DM-related care in the perioperative period.

SUPPLEMENTAL ONLINE MATERIAL
Supplemental material can be found online at http://www.mcpiqojournal.org. Supplemental material attached to journal articles has not been edited, and the authors take responsibility for the accuracy of all data. Grant Support: The work was supported in part by the Mayo Clinic Robert D. and Patricia E. Kern Center for the Science of Health Care Delivery and a grant from the National Institutes of Health (NIH) (grant number R01 GM105688-01). The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH. The funding sources for this work had no role in the study design; in the collection, analysis, and interpretation of data; in the writing of the report; or in the decision to submit the article for publication.