{'fraction_non_alphanumeric': 0.031814146197099734, 'fraction_numerical': 0.029397257571273552, 'mean_word_length': 5.234624846248463, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 4, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Evaluation of ICD-10 algorithms to identify hypopituitary patients in the Danish National Patient Registry

Objective Routinely collected health data may be valuable sources for conducting research. This study aimed to evaluate the validity of algorithms detecting hypopituitary patients in the Danish National Patient Registry (DNPR) using medical records as reference standard. Study design and setting Patients with International Classification of Diseases (10th edition [ICD-10]) diagnoses of hypopituitarism, or other diagnoses of pituitary disorders assumed to be associated with an increased risk of hypopituitarism, recorded in the DNPR during 2000–2012 were identified. Medical records were reviewed to confirm or disprove hypopituitarism. Results Hypopituitarism was confirmed in 911 patients. In a candidate population of 1,661, this yielded an overall positive predictive value (PPV) of 54.8% (95% confidence interval [CI]: 52.4–57.3). Using algorithms searching for patients recorded at least one, three or five times with a diagnosis of hypopituitarism (E23.0x) and/or at least once with a diagnosis of postprocedural hypopituitarism (E89.3x), PPVs gradually increased from 73.3% (95% CI: 70.6–75.8) to 83.3% (95% CI: 80.7–85.7). Completeness for the same algorithms, however, decreased from 90.8% (95% CI: 88.7–92.6) to 82.9% (95% CI: 80.3–85.3) respectively. Including data of hormone replacement in the same algorithms PPVs increased from 73.2% (95% CI: 70.6–75.7) to 82.6% (95% CI: 80.1–84.9) and completeness decreased from 94.3% (95% CI: 92.6–95.7) to 89.7% (95% CI: 87.5–91.6) with increasing records of E23.0x. Conclusion The DNPR is a valuable data source to identify hypopituitary patients using a search criteria of at least five records of E23.0x and/or at least one record of E89.3x. Completeness is increased when including hormone replacement data in the algorithm. The consequences of misclassification must, however, always be considered.


Introduction
Hypopituitarism is defined as deficiency of one or more hormones from the anterior or posterior pituitary gland. Common causes are pituitary adenomas and their treatment, radiotherapy of intracranial tumors and surgery near the pituitary gland. The clinical presentation of hypopituitarism is heterogeneous and varies according to type and extent of hormonal loss. However, it is associated with increased morbidity 1 and mortality. 2,3 Routinely collected health data are valuable for many different purposes including administrative health system planning and epidemiologic surveillance studies determining disease outcome over time. 4 Denmark is unique in having numerous continuously updated population-based health registries with a high level of completeness, where the individuallevel data recording and unique personal identifiers provide an invaluable opportunity to link registries and follow patients over time. 4,5 Further, it allows identification of specific submit your manuscript | www.dovepress.com

76
Berglund et al cohorts rapidly. Nevertheless, using routinely recorded health data, the risk of misclassification should always be considered. Thus, studies evaluating codes and algorithms (combinations of routinely collected health data) used to identify patients with a specific condition are a prerequisite for using routinely collected data in epidemiologic research. 6,7 To our knowledge, no study has reported the validity of the International Classification of Diseases (10th edition [ICD-10]) diagnoses of hypopituitarism in administrative health registries. We conducted a study aiming to estimate the positive predictive value (PPV) as well as completeness of ICD-10 diagnoses of hypopituitarism in the Danish National Patient Registry (DNPR). Based on experience from a previous Danish registry study on growth hormone deficiency (GHD), 8 we expected relatively low PPVs of the ICD-10 diagnoses of hypopituitarism. Previous studies from the DNPR evaluating ICD diagnoses of acromegaly 9 and craniopharyngiomas 10 reported low PPVs, which however were improved using algorithms for the searches. An algorithm-based search for patients with nonfunctioning pituitary adenomas in The Swedish National Patient Registry showed a PPV as high as 91%. 11 Therefore, we also aimed to develop and evaluate algorithms for identification of hypopituitary patients in the DNPR using combinations of ICD-10 diagnoses and data of hormone replacement therapy.

Methods Setting
Denmark is divided into five administrative regions. The Central Denmark Region has ~1.3 million inhabitants and accordingly 23% of the total Danish population. Inhabitants are primarily referred to hospitals in their home region, thus it can be assumed that the source population for this study included all inhabitants in the Central Denmark Region. The health care system in Denmark is a public tax-funded system ensuring all citizens equal access to health care. Since 1968, all persons residing in Denmark have been assigned a unique 10-digit civil registration number from which date and year of birth, as well as gender, can be identified. The civil registration number enables accurate matching of data from different data sources.

The DNPR
The DNPR contains data on all nonpsychiatric admissions to hospitals since 1977, and likewise data on all outpatient contacts since 1996. Among included data are dates of admissions and discharges, discharge diagnoses, information of the institutional origin of the discharge diagnoses as well as surgery and treatment procedures. Diagnoses are classified according to the International Classification of Diseases (eighth edition until 1993 , and thereafter ICD-10).

Hormone replacement
Hypopituitary patients in Denmark are offered free of charge hormone replacement administered by hospital pharmacies or by endocrine or pediatric departments. We searched the hospital pharmacy database at Aarhus University Hospital, which covers all administrations of hormones from Aarhus University Hospital during 1997-2014 and all hospitals in Central Denmark Region from 2014 and onward, on following Anatomical Therapeutic Chemical Classification System codes: H01AC01 (somatropin), H02AB09 (hydrocortisone), H03AA01 (levothyroxine sodium), H03AA02 (liothyronine sodium), H01BA (vasopressin analogous) and G03B, G03C and G03F (sex hormones).
Data of hormone replacement administered by peripheral hospitals in Central Denmark Region before 2014 were retrieved from endocrine and pediatric departments. Combined, data were retrieved for 664 patients in the candidate cohort, hereof 657 patients from the dataset retrieved from the hospital pharmacy at Aarhus University Hospital.

Medical record evaluation
The candidate population was identified in January 2013 searching the DNPR for all individuals recorded at least once, at any age, from January 1, 2000, to December 31, 2012, with a primary or secondary ICD-10 diagnosis of hypopituitarism or a diagnosis presumed to be associated with an increased risk of being hypopituitary (Table 1). Based on this dataset, the medical records were identified and reviewed.
Criteria for the diagnosis of hypopituitarism are shown in Table 2. The primary review of medical records was done electronically, and when insufficient to make a definite confirmation or disproval of hypopituitarism, paper records were identified at one or more departments. If a diagnosis of hypopituitarism could not be confirmed after thorough record review, the status of the patient was considered nonhypopituitary. When in doubt, a senior specialist in endocrinology and hypopituitarism was consulted, and status was decided by consensus. In the following, those patients with verified hypopituitarism according to this method are named confirmed patients. Besides the status of hypopituitarism, the cause of hypopituitarism was assessed and registered during the medical record review as well. The medical record review took place from October 2014 to April 2016.

Data analysis
Validation metrics included PPVs and completeness using the medical record review as reference standard. The PPV was defined as the proportion of confirmed patients among all patients in the DNPR with records of candidate ICD-10 diagnoses. Analyses were stratified by gender, type of diagnosis (primary or secondary), age at onset (childhood onset [CO] or adulthood onset [AO]) of hypopituitarism, use of hormone replacement, and by year of first diagnosis (before or after 2000). CO and AO hypopituitarism were defined as first year of registration with a candidate ICD-8 code (Table 3) or a candidate ICD-10 code before or after an age cutoff of 18 years. Year of first diagnosis was defined as the first year of registration with a candidate ICD-8 or ICD-10 code or first year of registration with E23.0x or E89.3x.
Completeness of ICD-10 diagnoses and algorithms were computed as the proportion of identified patients in the DNPR out of all confirmed hypopituitary patients.
Two main types of algorithms were evaluated: 1) algorithms based on ICD-10 diagnoses of hypopituitarism only and 2) algorithms based on ICD-10 diagnoses of hypopituitarism combined with data of hormone replacement. Both types of algorithms were evaluated for at least one, three or five records of E23.0x (hypopituitarism, and/or at least one record of E89.3x (post-procedural hypopituitarism). E23.0x and E89.3x were selected for the algorithms, as they displayed the most favorable compromise between PPV and completeness during initial explorative analyses.
All evaluations were made using a dataset retrieved from the DNPR May 2016. Prevalence was estimated as the number of confirmed hypopituitary patients divided by the average number of inhabitants in Central Denmark Region during 2000-2012 (http://www.noegletal.dk). Confidence intervals (CIs) were estimated as 95% CIs. All analyses were made in StataCorp 13.1.

Ethics
The study was approved by the Danish Data Protection Agency (journal number: 2012-41-0149) and by the Danish

79
Identification of hypopituitary patients in routinely recorded health data had 54.5% (85/156) hormone-or nonhormone producing adenomas, including prolactinomas, or idiopathic hypopituitarism as cause of hypopituitarism, while the remaining had a wide spectrum of causes of hypopituitarism. Of confirmed patients identified by the same algorithm had 39.2% (296/755) idiopathic hypopituitarism or hypopituitarism due to hormone-or nonhormone producing adenomas, according to the medical record review.

Discussion
In this population-based study, the validity of ICD-10 diagnoses of hypopituitarism in the DNPR during 2000-2012 was assessed using medical record review as reference standard. Alone, neither E23.0x (hypopituitarism) nor E89.3x (postprocedural hypopituitarism) had sufficient validity for identification of hypopituitary patients in the DNPR. However, an algorithm including at least five records of E23.0x and/or at least one record of E89.3x yielded an acceptable compromise between PPV and completeness. Completeness was even further improved without a concurrent loss in PPV when a criterion of hormone replacement was applied to the algorithm.  The PPV of E23.0x (hypopituitarism) was 73.3% (95% CI: 70.4-76.1) and the PPV of E89.3x (postprocedural hypopituitarism) was 88.9% (95% CI: 85.5-91.7). The PPVs for all candidate ICD-10 diagnoses are shown in Table 5. Of all confirmed patients (n=911), 90.8% (n=827) had at least one record of E23.0x or E89.3x.
The PPVs for algorithms identifying patients recorded at least one, three or five times with E23.0x and/or at least once with E89.3x ranged from 73.3% (95% CI: 70.6-75.8) to 83.3% (95% CI: 80.7-85.7) ( Table 6). Including data of hormone replacement to the algorithms, PPVs increased with an increasing number of records of E23.0x as well. Completeness decreased for both types of algorithms, when the number of records of E23.0x was increased. However, the loss of confirmed patients was lower for the algorithms including hormone replacement (Table 6). PPVs were not improved for any of the algorithms when stratifying on CO and AO patients, respectively.
One-hundred and fifty-six confirmed patients were not identified by the algorithm that included at least five records of E23.0x and/or at least one record of E89.3x, and of those  In general, the accuracy of clinical coding may relate to how closely a clinical condition reflects a relevant code, with the lowest accuracy relating to rare, ill-defined or complex conditions. 12 Comparing Danish validation studies on more or less complex conditions, 9,[13][14][15] this may also apply to the coding in the DNPR. In this study, the accuracy of postprocedural hypopituitarism (E89.3x) was distinctly superior to hypopituitarism (E23.0x). This difference possibly reflects the inbuilt specification of prior cranial surgery or irradiation in E89.3x, as these are procedures that per se increase the risk of hypopituitarism, and thus may the diagnostic code E89.3x reflect the clinical condition more accurately.
The accuracy of coding in the DNPR in the present study was also related to the year of incidence (before or after 2000), with the lowest accuracy in the later period ( Table 4). The more accurate coding before 2000 may be due to diagnosis of primarily severely affected patients at that time, making subsequent confirmation of hypopituitarism during medical record review easier. However, it may also reflect less optimal coding practice in the later period, with more patients being coded as hypopituitary without truly being hypopituitary.
Comparing men and women, a more accurate coding was observed in men. In one of our previous studies on GHD, where a similar approach as in the present study was used for identification of GHD patients, significantly more men than women were verified with GHD as well. 8 Women have significantly more visits at their general practitioner than men (www.dst.dk), and we therefore speculate whether women with, e.g., pituitary adenomas are detected and referred to hospital by their general practitioner with smaller adenomas than men subsequently reducing the risk of postprocedural hypopituitarism. Further, postmenopausal women may receive a diagnosis of gonadotropin deficiency less likely than men may.
PPVs increased for the algorithms with increasing number of records of E23.0x (hypopituitarism). This complies with findings in similar studies, which, similar to hypopituitarism, require continued follow-up. 9,15 Thus, it is possible that searches including repeated coding with specific ICD diagnoses will increase the accuracy of the data found in the DNPR for patients with other chronic conditions requiring continuing hospital follow-up as well.
Approximately 10% (9.2%, 84/911) of the confirmed hypopituitary patients had no records of E23.0x (hypopituitarism) or E89.3x (postprocedural hypopituitarism). Therefore, some underreporting of hypopituitarism is present, and it cannot be ruled out that it is even more prevalent than here. However, the prevalence of hypopituitarism was estimated to be 74.5 per 100,000. This estimate is ~50% higher than the only previously reported estimate of hypopituitarism of 45.5 per 100,000 in a study from Spain, 16 and therefore it seems likely that only very few diagnosed hypopituitary patients, if any, have been missed, when the candidate population was identified. It should, however, be stressed that the prevalence of 74.5 per 100,000 is a minimum estimate since some hypopituitary individuals may remain undiagnosed. Further, some may have received a diagnosis from their general practitioner, thus were never recorded in the DNPR with an ICD-10 diagnosis of hypopituitarism or related pituitary disease. Nevertheless, we believe that this is rare. In general, an observed under-or overreporting of a certain disease could be due to discrepancy between diagnostic criteria used in the daily clinic and diagnostic criteria used in a study. In the present study, however, the diagnostic criteria of hypopituitarism are in accordance with the clinical guidelines of hypopituitarism from the Endocrine Society. 17 Searching the candidate population for individuals with a registration of hormone replacement, the PPV was expectedly high (89.3%), but completeness was poor (65.1%). The data on hormone replacement therapy retrieved from the pharmacy at Aarhus University Hospital were the only data uniformly collected during the entire period 2000-2012, where the candidate population was identified. By contrast, data retrieved from endocrine and pediatric departments at the peripheral hospitals in the Central Denmark Region were of poor quality, since they had not been collected uniformly, and a considerable proportion of data had been destroyed. The poor quality was mirrored by the fact that only seven additional hormone replaced patients were identified at our search at the peripheral hospitals. In 2014, a uniform electronic registration of hormones administered directly to patients by hospital pharmacies in Central Denmark Region was established, and thus completeness is believed to increase for future populations of hypopituitary patients. However, some departments at the peripheral hospitals still retrieve hormones from the peripheral hospital pharmacies for administration to patients without an individual-level registration at the pharmacy. Part of the low completeness observed, searching the candidate population for patients receiving hormone replacement, could also be due to an incorrect prescription procedure of hormones by the treating physician, but we believe that this is rare, thus having only a minor influence on the estimate.
Hypopituitary patients constitute a very heterogeneous group of patients as the type and extent of hormonal loss differ among patients, and furthermore the etiology of Clinical Epidemiology 2017:9 submit your manuscript | www.dovepress.com

81
Identification of hypopituitary patients in routinely recorded health data hypopituitarism contributes to the heterogeneity. A larger proportion of the confirmed patients not identified by the algorithm (including at least five records of E23.0x and/or at least one record of E89.3x; 54.5%) had either idiopathic hypopituitarism or hypopituitarism due to adenomas or prolactinomas than confirmed patients that were identified by the algorithm (39.2%). An association between mortality and cause of hypopituitarism has been demonstrated, and mortality was, e.g., lower in hypopituitary patients due to adenomas than due to craniopharyngeomas or cancer. 3,18 Considering this, the difference between algorithm-identified and algorithm-unidentified patients may indicate that high-risk patients are more likely to be correctly coded in the DNPR.
Some limitations must be considered when interpreting the results of this study. First, the study is based on findings from one Danish region only and as such, we cannot account for the remaining regions. An optimal study design would include evaluation of the algorithms on another study population in another region. Due to the large number of additional medical records to be reviewed such a design was however considered unfeasible, and considering that Denmark is a welfare state with only small demographic variations between regions combined with the uniform public health system, we find it reasonable to consider results from one region representative for the entire country. A second limitation is related to the verification process applied to the study population, as the reviewer was nonblinded to the ICD-10 diagnoses of the patients. However, we find it unlikely that this should have had any major influence on our study results.
In this study, including ~1.3 million inhabitants in Central Denmark Region during 2000-2012, the DNPR is found to be a valuable source to identify hypopituitary patients if an algorithm-based search is used. We recommend use of an algorithm including at least five records of hypopituitarism and/or at least one record of postprocedural hypopituitarism. Completeness can be improved by adding a criterion of at least one record of hormone replacement due to hypopituitarism to the algorithm.