{'fraction_non_alphanumeric': 0.03676648639922512, 'fraction_numerical': 0.021107434014044718, 'mean_word_length': 5.134934389700421, 'pattern_counts': {'":': 0, '<': 2, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Cognitive variability—A marker for incident MCI and AD: An analysis for the Alzheimer's Disease Neuroimaging Initiative

Introduction The potential of intra-individual cognitive variability (IICV) to predict incident mild cognitive impairment (MCI) or Alzheimer's disease (AD) was examined and compared to well-established neuroimaging and genetic predictors. Methods IICV was estimated using four neuropsychological measures for n = 1324 Alzheimer's Disease Neuroimaging Initiative (ADNI) participants who were cognitively healthy or diagnosed with MCI at baseline. IICV was used to predict time to incident MCI or AD, and compared to hippocampal volume loss and APOE ε4 status via survival analysis. Results In survival analyses, controlling for age, education, baseline diagonosis, and APOE ε4 status, likelihood ratio tests indicate that IICV is associated with time to cognitive status change in the full sample (P < .0001), and when the sample was restricted to individuals with MCI at baseline (P < .0001). Discussion These findings suggest IICV may be a low-cost, noninvasive alternative to traditional AD biomarkers.


Introduction
The prevalence of Alzheimer's disease (AD) is rising, creating an urgency to develop effective interventions [1,2]. Current strategies include: (1) intervention during the protracted presymptomatic or preclinical stages; and (2) development of practical and effective means to prevent the disease-associated suffering and untenable costs. A prevention-focused approach necessitates the identification of biological indicators of disease process, that is, biomarkers [3,4].
Major efforts, including the Alzheimer's Disease Neuroimaging Initiative [5] (ADNI) and the Australian Imaging, Biomarkers and Lifestyle [6] (AIBL) Study of Aging, have expanded our understanding of preclinical and subclinical stages of AD and what biomarkers might be used to detect disease processes well before the onset of clinical symptoms. The most promising biomarkers are obtained from cerebrospinal fluid and brain imaging [7]. However, given the difficulties in disseminating collection methods outside of research centers, and the arduous and invasive nature of some collection procedures, there is a desire to develop noninvasive, convenient markers [8,9]. The development of a cognitive marker, once established and validated, would offer an alternative for individuals unable or unwilling to submit to the collection of traditional biomarkers.
One such proposed cognitive marker, intra-individual cognitive variability (IICV), estimates variability between cognitive domains measured at one time-point. Overall, researchers have used two conceptual methods to investigate the prognostic value of cognitive variability. The first method being an examination of variability across domains at one time (dispersion) [10][11][12][13][14], and the second being variability across trials (inconsistency) administered either in one session or over time [15][16][17][18][19][20] or both dispersion and inconsistency [17,19].
Holtzer et al. [10] and many others [13,14,20] have examined the usefulness of a dispersion-based IICV to predict cognitive decline and incident AD. This approach is not new. For example, significant disparity between verbal and performance intelligence quotients (IQs) is a longestablished correlate with underlying neuropathology [21,22]. In Holtzer et al. [10], IICV was estimated as the degree to which an individual's test scores differed from their mean standardized test performance. The investigators found greater variability in performance (dispersion) was associated with increased risk for dementia a decade later. This suggested IICV, like traditional biomarkers, might cooccur with preclinical brain alterations.
We hypothesized that IICV would predict incident AD and mild cognitive impairment (MCI), and that IICV would demonstrate strong criterion validity, estimated by comparing IICV with an established neuroimaging biomarker, hippocampal volume loss (HVL), and with apolipoprotein ε4 allele (APOE ε4) a genetic risk marker. Our overall goal was to explore the utility of IICV as a potential marker of preclinical cognitive changes and examine whether baseline IICV predicted subsequent incident cognitive endpoints, including MCI and AD.

Study design
Using an ex-post facto design and using an estimate of IICV used by Holtzer et al. [10] as our primary predictor, we examined the association of IICV and conversion to MCI or AD in an ADNI sample, including adults who were cognitively healthy or diagnosed with MCI at baseline. We repeated our analyses with an MCI sub-sample. Finally, we examined the contribution of IICV as a predictor of incident MCI and AD when HVL and a genetic risk factor, APOE ε4 status, were also included in the models.

ADNI
Data used in the preparation of this article were obtained from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database (adni.loni.usc.edu). ADNI was launched in 2003 as a public-private partnership, led by PI Michael W. Weiner, MD. The primary goal of ADNI has been to test whether serial magnetic resonance imaging (MRI), positron emission tomography, other biological markers, and clinical and neuropsychological assessment can be combined to measure the progression of MCI and early AD (adni-info.org [23]).

Participants
Data were collected at ADNI study centers and clinics across the United States and Canada from three ADNI funding cycles (ADNI 1, ADNI 2, and ADNI GO) [5,24,25]. ADNI eligibility criteria included the following: age 55 to 90 years; English or Spanish language speakers; no diagnosis of depression; and baseline diagnosis of early AD, MCI, or cognitively normal (CN). Cognitive status was confirmed with designated cut off scores for the Clinical Dementia Rating Scale, mini-mental state examination, and Wechsler Memory Scale Logical Memory II. A complete account of ADNI exclusion criteria can be found at www.adni-info.org [23]. Evaluations were repeated every 6 months (ranging from 6 to 72 months), with a mean total follow-up time of 30.81 months (SD 5 23.85). The results of cognitive assessments, physical examinations, and MRI scans were considered in determining diagnostic status [24,25]. Supplementary Material describes how our primary outcome, diagnostic conversion was determined.
Before application of exclusion criteria in our study, the total subject pool included 1729 participants. We excluded subjects if they completed fewer than two visits, had incomplete or missing neuropsychological data, or carried a diagnosis of AD at baseline. After exclusionary criteria were applied, 1324 participants remained in the sample (see Fig. 1). For the MCI subgroup analyses, individuals who were CN at baseline were excluded, resulting in a sample of 825 individuals.

Estimate of cognitive variability
We sampled the following cognitive domains to determine IICV: Attention, processing speed, executive functioning, working memory, and verbal memory. In total, four index scores from three tests were used to calculate participants' IICV score. Specific indices included: Rey Auditory Verbal Learning Test, Total of Learning Trials, American National Adult Reading Test (ANART), and Trail Making Test (TMT) A and B. Because lower scores on TMT A and B are indicative of better functioning (faster performance), scores were reversed before z transforming them.
The single time-point IICV index was calculated per a published algorithm [10]. Briefly, individual test scores were standardized using score distributions based on the sample obtained after inclusion and exclusion criteria were applied (n 5 1324). For each participant, individual test z-transformed scores were compared to their mean test z-score, resulting in a summation of cross-test variability. Consistency between test scores, regardless of value, will demonstrate low levels of IICV, whereas extreme differences between test scores will result in high IICV.

Hippocampal volume loss
Measurements of hippocampal volume were conducted per ADNI protocols [25]. HVL represented the absolute difference between bilateral baseline volume measurement and Fig. 1. CONSORT diagram. The terms ADNI 1, ADNI GO, and ADNI 2 each refers to groups of participants who were compiled into one group and included in the study from different funding cycles of the Alzheimer's disease neuroimaging initiative abbreviations. The term converters are those who were noted as changing in diagnosis from either cognitively normal (CN) or mild cognitive impairment (MCI) to MCI or Alzheimer's disease (AD) over the course of the study. Nonconverters are those who did not change in diagnosis over the course of the study. the measurement occurring closest to time of conversion, if subject converted, or at the last available measurement if subject did not convert. The value was divided by time between measurements. This rate of volume loss was divided by baseline intracranial volume to adjust for individual differences. Thus, in models presented here, HVL represents an estimate of the proportional rate of HVL adjusted for intracranial volume.

Statistical analysis
Baseline demographic characteristics of converters and nonconverters were compared using t tests for continuous variables and chi-square tests for categorical variables, using SPSS Statistics for Windows, version 23.0. Cox proportional hazard survival models were used to investigate associations between IICV and time-to-conversion, controlling for age, education, APOE ε4 status, stratifying on baseline diagnosis (MCI or CN), and using a likelihood ratio test (LRT) comparing survival models with and without IICV. IICV tertile survival curves were estimated by including tertiles as the only covariate in the model. To investigate relative performance of IICV versus APOE ε4 status and HVL, two methods were used. First, concordance increases over a "base" survival model (age, education, and stratified baseline diagnosis) was examined when adding IICV, APOE ε4 status, or HVL. Second, in a model including all the "base" items, regression coefficient estimates were compared. Survival model analyses were conducted using R for Windows, version 3.2.3, and "survival" package, version 2.38-3.
The above analyses were performed for both the full group and an MCI subgroup. This was done to investigate if the association between IICV and conversion was markedly different when the sample was restricted to a subgroup (MCI). Additionally, all analyses were performed for both continuous variables of IICV and HVL, and tertiles of IICV and HVL, to assess effects of discretizing these variables. For continuous IICV and HVL, both were standardized (mean of 0, standard deviation of 1) before inclusion into models.
To support assumptions of proportionality, the MCIsubgroup survival models incorporated education as a discrete covariate with three categories: ,12 years (category 1), 12 years to ,16 years (category 2), and 16 years (category 3). In survival models with both MCI and CN, education was included as a continuous covariate. Given concern for nonproportionality in models with HVL tertiles, analyses focused on models where IICV and HVL were continuous covariates. Both continuous and tertiles of IICV and conformed well to proportionality assumptions.
Product-limit survival curve estimates comparing time to conversion between IICV tertiles for individuals who were CN or diagnosed with MCI at baseline (n 5 1311) revealed that the group with the smallest magnitude of IICV had an estimated median survival time .96 months (CI unavailable). Individuals in a moderate IICV range had an estimated median survival time of .96 months also (95% lower CI limit 84 months; upper limit unavailable). Individuals with the highest degree of IICV had an estimated median survival time of 48 months (95% CI, 36-72 months). Survival times among the IICV tertiles differed as indicated by a log-rank test (c 2 5 71, df 5 2, P , .0001). Fig. 2 portrays product-limit survival estimate curves for the full group analysis. LRTs were used to examine the association of survival time with the covariates education and age after accounting for continuous IICV and APOE ε4 status. Education (c 2 5 0.0004, df 5 1, P 5 .9814) was not statistically significant, whereas age (c 2 5 5.892, df 5 1, P 5 .0152) and IICV (c 2 5 42.6, df 5 1, P , .0001) both were statistically significant.
We found similar results when the survival analysis was repeated in a sample restricted to individuals with MCI at baseline. Survival analysis revealed that for this subgroup, participants with the lowest level of IICV had an estimated median survival time .96 months (95% lower CI limit: 84 months, upper limit unavailable); those with a moderate level of IICV had an estimated median of 72 months (95% lower CI limit: 48 months, upper limit unavailable); and subjects with the highest variability index had an estimated median of 36 months (95% CI, 36-48 months). Survival times between IICV tertiles were statistically different, (c 2 5 67.6, df 5 2, P , .0001) by a log-rank test. See product-limit survival estimate curve in Fig. 3. In addition, LRTs showed that age (c 2 5 3.6305, df 5 1, P 5 .1668) and education (c 2 5 0.3714, df 5 2, P 5 .8305) were not statistically significant in the MCI subgroup after accounting for continuous IICV and APOE ε4 status; IICV (c 2 5 49.3, df 5 1, P , .0001) remained significant.
We examined how predictive IICV is after accounting for all other variables for the full sample. IICV was significantly associated with conversion. The hazard ratio (HR) for 1 standard deviation (SD) increase in IICV is 1.20 (95% CI, 1.08-1.35; P 5 .0010), compared to a HR for a 1 SD increase in HVL of 1.97 (95% CI, 1.72-2.26; P , .0001), and a HR of 1.83 (95% CI, 1.37-2.46; P 5 .0005) for APOE ε4 positive status compared to negative. The "base" model for the full group has a concordance of 0.542. In comparison, adding continuous IICV to the base in a separate model increases concordance to 0.645, adding continuous HVL to the base as a separate model instead increased it to 0.714, and adding APOE ε4 status to the base model increased concordance to 0.637. Table 3 includes beta weights, HRs, and CIs for IICV tertile, HVL tertile, and APOE ε4 status.
An examination of a subgroup whose baseline diagnosis was MCI revealed similar findings when all variables were entered into the model simultaneously. The HR for 1 SD increase in IICV was 1.21 (CI, 1.08-1.36; P 5 .0015) compared to HR for 1 SD increase in HVL 5 1.99 (CI, 1.73-2.29, P , .0001), and positive APOE ε4 status HR of 1.78 (CI, 1.28-2.48, P 5 .0007). All were strongly associated with incident AD. The "base" model for the MCI subgroup's concordance was 0.526. In comparison, adding continuous IICV to the base model increased concordance to 0.663; adding continuous HVL to the base model instead increased it to 0.732; and adding APOE ε4 status to the base model instead increased it to 0.636. See Table 3 for beta weights, HRs, and CIs for the model restricted to individuals whose initial diagnosis was MCI. All analyses were conducted with IICV included as a categorical variable and a continuous variable. Results were highly similar in either case. Discrete variable results are provided in Supplementary  Table 1.

Discussion
Overall, we found IICV estimates obtained from a single evaluation predicted incident MCI and AD in a large sample of well-characterized older adults enrolled in ADNI studies. Specifically, we found individuals exhibiting the highest level of variability in performance on three neuropsychological measures (IICV) demonstrated shorter times to conversion in cognitive status compared to individuals in the lowest tertile of variability. Differences in conversion rates emerged approximately 30 months after initial testing and remained evident through the last available observation, approximately 5 years after IICV was measured at baseline. Logrank survival analyses indicated that after accounting for age, education, HVL, and APOE ε4 status, the association between IICV and time to conversion remained statistically significant. Altogether, these data provide support for this practical and noninvasive cognitive AD marker.
At present, researchers rely on resource-intense biomarkers, which are well validated but invasive [26,27]. For example, amyloid beta 1-42 (Ab 1-42 ), total tau protein (ttau) and phosphorylated tau (P-tau) are robust biomarkers of disease pathology and clinical course [28,29]. These predictors can only be obtained with a lumbar puncture (LP) or neuroimaging of cerebral amyloid plaques, requiring injection of a radioactive compound, and access to unique scanning expertise [30]. At minimum, other biomarkers entail specialized neuroimaging and often require longitudinal follow-up [8,9]. For example, MRI measurement of HVL is highly accurate in predicting conversion from MCI to AD [31,32]; however, volume loss can only be measured with repeated evaluations. Altogether, traditional AD biomarkers are inarguably important, but prohibitively expensive; often dependent on repeated evaluations; contraindicated for some subjects [33]; and cumbersome to implement, especially in nonacademic centers. Data from the present study suggest that IICV is among the possible utilitarian alternatives to traditional biomarkers.
When determining which neuropsychological indices to include in our IICV estimate, care was taken to detect expected dissociations between fluid intelligence (e.g., executive function) and crystallized intelligence (e.g., semantic knowledge), and early impairments in hippocampal-based learning to maximize the assessment of patterns relevant for AD. For example, the ANART reflects crystallized intelligence, and is typically stable with aging and in the presence of early to mid-stage disease [34]. In contrast, the AVLT, a measure of verbal memory, is sensitive to hippocampal-based learning, reflecting a very early cognitive change in AD [35,36]. Although Salthouse and Soubelet [14] cautioned against the use of measures reflecting cognitive abilities acquired in early life, e.g., vocabulary, we intentionally selected a test used to assess baseline intelligence so it could be contrasted against a score reflecting disease-related changes. Given that a change from baseline performance is a diagnostic criteria for dementia due to AD [7] and early changes are often in the domain of memory, we sought to detect when hippocampally based memory markedly differed from one's estimated baseline abilities, believing this may be an early subtle sign of disease. Additionally, tests such as Trails A and B were chosen as contrasting measures of executive function. Individuals with AD and MCI often demonstrate a characteristic dissociation in performance between Trails A and Trails B. Specifically, Trails A performance generally remains relatively well-preserved, being dependent on intact motor speed, visual scanning, and simple attention. On the other hand, the more challenging Trails B reveals even subtle executive dysfunction, with its additional demands on working memory and dual-task performance [37]. Thus, measures were selected based on their sensitively to AD-related performance variations, that is, the peaks and valleys of a cognitive profile occurring early in the disease process. Certain measures sensitive to AD-related cognitive changes (e.g., category fluency, digit symbol tasks) were excluded if they did not have an obvious contrasting task. For example, Trails A is contrasted to Trails B. This was done in an effort to minimize the number of measures needed to estimate variability and support dissemination.
Holtzer et al. [10] and a number of others have examined the utility of cognitive variability [11,12,20,38]. Although methods for estimating the scatter or dispersion of cognitive test scores vary, previous analyses point toward the conclusion that variability across test indices is a marker for brain dysfunction, be it due to increasing dementia severity [39], mental illness [40], or head injury [41]. Some have speculated that intra-individual variability is related to disease-associated disruptions in neural networks and functional connectivity [42], whereas others relate the association to reductions in the ability to sustain mental efforts, especially executive functioning [43].
Our study replicated and expanded previous findings. In particular, our inclusion of APOE ε4 status and HVL provides unique insights. Not surprisingly, APOE ε4 status and HVL predicted conversion in our models. Importantly, IICV remained a significant predictor of time to conversion even after accounting for the influence of APOE ε4 status and HVL. Altogether, IICV provided unique predictive information.
There were limitations to the study. First, subjects enrolled in ADNI are predominantly non-Hispanic and white. In addition, the ADNI sample reflects only those individuals who were able and willing to have a complete set of invasive biomarkers measured. Altogether, this limits the generalizability of findings. Another limitation is that IICV, by diagnostic definition, would be greater for individuals with MCI, which presents with a greater risk for conversion to AD than CN status. In an attempt to address the potential confound, we (1) adjusted for baseline diagnosis and (2) conducted a second analysis, restricted to subjects with MCI at baseline. Given the low rate of conversion in the CN sample, it was not possible to examine an exclusively CN sample. Another potential limitation was our operationalization of IICV. We used a dispersion model to compare multiple domains at one time-point in an attempt to investigate the potential of a clinically relevant marker specific to AD. This may limit the ability to use the marker in more heterogeneous samples where mixed dementia may be common.
Although validation in other populations and cohorts is needed to corroborate our findings, IICV could provide a practical alternative to traditional biomarkers. Many emerging prevention strategies rely on an assessment of risk to identify individuals likely to benefit from early intervention, necessitating collection of biomarkers before drug administration. Consequently, if replicated, the findings would have a number of research and clinical practice implications. For example, IICV could offer an alternative or adjunct to traditional biomarkers in settings and groups not amenable to LPs and neuroimaging. In rural settings, a cognitive marker could be widely disseminated with minimal time and equipment demands. Furthermore, there are cultural groups who mistrust medical procedures based on historical maltreatment and discrimination, for example, African Americans [44], for whom cognitive testing may be more acceptable than the collection of traditional biomarkers, including LP. Overall, IICV could be considered an alternative to relying on more invasive biomarkers to identify target populations and a low-cost, easily administered means to assess risk for incident MCI and AD, allowing for a broader ability to identify at-risk individuals than traditional biomarkers.

Conclusions
Although these findings require replication and further validation, this study provides support for the use of IICV as a novel predictor of incident MCI and AD and practical alternative to traditional AD biomarkers. IICV is particularly appealing in that it could be widely disseminated with minimal time and equipment demands.

Acknowledgments
This article is an update and expansion of E. Anderson's dissertation. As such he would like to thank the following: The University of Wisconsin-Madison, as well as Dr. Ruth Lynch, Dr. Fong Chan, and Dr. David Rosenthal, all of whom served on his committee. There are no known conflicts to report. Each of the authors in this study is paid a salary by their respective institutions and/or universities; this study was conducted without any additional funding from grants or outside sources. Contributions by coauthors, S.C.J. and N.M.D., and senior author C.E.G. are supported by funding from the NIH-NIA for the Wisconsin Alzheimer's Disease Center (P50 AG033514). Data collection and sharing for this project was funded by the Alzheimer's Disease Neuroimaging Initiative (ADNI) (National Institutes of Health Grant U01 AG024904) and DOD ADNI (Department of Defense award number W81XWH-12-2-0012). ADNI is funded by the National Institute on Aging, the National Institute of Biomedical Imaging and Bioengineering, and through generous contributions from the following: AbbVie, Alzheimer's Association; Alzheimer's Drug Discovery Foundation; Ara-