{'fraction_non_alphanumeric': 0.024623115577889446, 'fraction_numerical': 0.005527638190954774, 'mean_word_length': 5.230046948356808, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Assessing Missing Data Assumptions in EHR-Based Studies: A Complex and Underappreciated Task.

Missing data are a pervasive problem in clinical and public health research, affecting both randomized trials and observational studies. Reasons for missing data can vary substantially across studies because of dropout or loss to follow-up, missed study visits, or an unrecorded measurement during an office visit. When faced with missing data, the best case scenario is simply a loss of statistical power. However, the study could suffer from selection bias, whereby the results’ external generalizability (ie, to the intended or target patient population) is compromised.1 Controlling for selection bias requires 2 tasks: researchers should (1) articulate their assumptions regarding the missingness of data that they are willing to believe and, (2) if necessary, use an appropriate statistical adjustment method.2-4 Regarding the first task, 1 of 3 assumptions are said to hold: missing completely at random (MCAR), missing at random (MAR), or missing not at random (MNAR). Intuitively, MCAR is reasonable when a data element is missing independent of any factor that is relevant to the primary analysis. For example, missing data may be considered MCAR in a trial conducted during the coronavirus disease 2019 (COVID-19) pandemic because of early trial termination after the implementation of a lockdown policy. However, in most instances, it is more plausible that missingness in a given data element is associated with 1 or more characteristics of the study participants, such as age. If all characteristics are measured and available for analysis, then the missing data may be MAR. However, if this is not the case, then the missing data may be considered MNAR, such as when a patient drops out in part because their health has deteriorated. Clear articulation of assumptions is critical because resolving selection bias hinges on them. If the missing data are deemed MNAR, then a definitive adjustment for selection bias will not be possible. However, despite its importance, articulation of missing data assumptions is seldomly reported, and descriptions of how missing data are handled often solely indicate the use of a particular analysis method (eg, inverse-probability weighting or multiple imputation). As such, readers often have no clear sense of why some data are missing and others not and, therefore, cannot gauge the external generalizability of the reported results. While this is a concern in all research, it is particularly problematic in studies that use electronic health records (EHRs) because data from these systems are generally not collected with a particular research agenda in mind. This has important and often unappreciated implications for dealing with potential selection bias. To illustrate, the Figure presents data for 6 patients from an EHR-based study of long-term outcomes following bariatric surgery among patients with type 2 diabetes at Kaiser Permanente Washington.5 The Figure highlights the complexity and heterogeneity of EHR-based data: some patients have rich data throughout the 2.5-year time window, while others have sparse data; some disenroll before the 2-year mark, while others remain enrolled; some have periods of intense interactions with the health care system, while others have very few interactions; some have numerous primary care visits during which no body mass index (BMI; calculated as weight in kilograms divided by height in meters squared) or hemoglobin A1c values were recorded. While this complexity would seem to have clear implications for any analysis, somewhat less obvious are the implications for assessing missing data assumptions. Missing data assumptions are typically considered through the lens of a missingness mechanism, which can be thought of as a joint decision that is implicitly made by all relevant parties Author affiliations and article information are listed at the end of this article.