{'fraction_non_alphanumeric': 0.04011461318051576, 'fraction_numerical': 0.019728498285499555, 'mean_word_length': 4.9386331938633194, 'pattern_counts': {'":': 3, '<': 6, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Stability of mismatch negativity event‐related potentials in a multisite study

Abstract Objectives Mismatch negativity (MMN), an auditory event‐related potential sensitive to deviance detection, is smaller in schizophrenia and psychosis risk. In a multisite study, a regression approach to account for effects of site and age (12–35 years) was evaluated alongside the one‐year stability of MMN. Methods Stability of frequency, duration, and frequency + duration (double) deviant MMN was assessed in 167 healthy subjects, tested on two occasions, separated by 52 weeks, at one of eight sites. Linear regression models predicting MMN with age and site were validated and used to derive standardized MMN z‐scores. Variance components estimated for MMN amplitude and latency measures were used to calculate Generalizability (G) coefficients within each site to assess MMN stability. Trait‐like aspects of MMN were captured by averaging across occasions and correlated with subject traits. Results Age and site accounted for less than 7% of MMN variance. G‐coefficients calculated at electrode Fz were stable (G = 0.63) across deviants and sites for amplitude measured in a fixed window, but not for latency (G = 0.37). Frequency deviant MMN z‐scores averaged across tests negatively correlated with averaged global assessment of functioning. Conclusion MMN amplitude is stable and can be standardized to facilitate longitudinal multisite studies of patients and clinical features.


| INTRODUCTION
Mismatch negativity (MMN) is an auditory event-related potential (ERP) component that is automatically evoked by an infrequently occurring "deviant" auditory stimulus that differs in duration, pitch, or another physical feature from a series of repeated preceding "standard" stimuli. MMN can be measured using either electroencephalography (EEG) or magnetoencephalography. It is believed to reflect sensory echoic memory, as detecting auditory deviance requires online formation and maintenance of a memory trace of immediately preceding standard stimuli. Because of its robust sensitivity to the pathophysiology of schizophrenia (Avissar et al., 2018;Erickson, Ruffle, & Gold, 2016), and its ability to predict conversion to a psychotic disorder in clinical high-risk (CHR) individuals (Bodatsch et al., 2011;Perez et al., 2014;Shaikh et al., 2012), MMN has great potential as an ERP biomarker in schizophrenia research, leading to its inclusion in the multisite North American Prodrome Longitudinal Study [NAPLS Addington et al. (2012)]. Accordingly, the test-retest reliability and stability of the MMN response over repeat test occasions must be evaluated to better understand the generalizability of this ERP component in multisite, clinical trials or longitudinal studies of psychosis.
Given the broad age range (12-35 years) in NAPLS (Addington et al., 2012), and potential age differences between CHR individuals who later transition to psychosis and those who do not, such studies must control for potential confounding effects of normal aging on MMN responses. One approach to adjust for any normal aging effects on MMN is to apply a simple linear regression model to the healthy control (HC) data. The resulting regression equation is used to calculate age-corrected MMN z-scores for all subjects. This is done by subtracting the predicted MMN based on a subject's age from his/her observed MMN score, and then dividing the result by the standard error of regression obtained from the HC age-regression model. Such age-corrected MMN z-scores derived from a HC model have no relationship with age in the HC sample, but any pathological age effects in patient or hold-out samples are preserved. The z-scores are readily interpretable as linear transformations of MMN raw scores, reflecting the degree of deviation or abnormality, in standard units, from the MMN expected for a person of a given age in the HC sample. This approach is not unprecedented, having been implemented previously in MMN (Biagianti et al., 2017;Perez et al., 2014), other ERP Hamilton, Woods, et al., 2019;Mathalon et al., 2018;Mathews et al., 2016;Perez, Woods, et al., 2012;, functional (Fryer et al., 2013;Fryer et al., 2016;Fryer et al., 2018) and structural (Heyes et al., 2001;Jernigan, Press, & Hesselink, 1990;Pfefferbaum et al., 1992) magnetic resonance imaging studies.
In any multisite study design comparing patients and controls, the effect of laboratory testing site should be carefully considered (Glover et al., 2012). Despite applying all possible best practices to minimize site-specific influences on experiments, differences in mean responses between sites (i.e., fixed effects of site) remain possible due to real differences in the random samples of participants studied at each site. Therefore, the site effect should be modeled and, even when site is not statistically significant, should not be disregarded based on this criteria alone. Furthermore, it may be of interest to test secondary/exploratory hypotheses that do not involve the entire study sample. In such cases, simply including site in the model may not be sufficient if the subset of participants is too small to accurately estimate site effects.
In NAPLS, the effect of site can be added as a categorical covariate in the age model. In this new model, a common age effect is estimated across all sites such that site effects on either MMN or age cannot create a spurious relationship between MMN and age. Moreover, the error from this new model can be used to calculate site-and age-adjusted MMN z-scores, reflecting the difference, in standard units, from the MMN expected for a person of a given age, from a particular site in the HC sample. Such z-scoring is particularly useful for planned comparisons between CHR individuals who convert (CHR-C) to a psychotic disorder and those who do not (CHR-NC) within 24 months of initial NAPLS baseline assessments because there is no feasible a priori method to match these subjects on age and/or sample an equal number from each site.
Accordingly, the goals of this study were to (a) describe a set of site and age regression models in HCs that will be used to create standardized, site-and age-adjusted MMN z-scores for all NAPLS subjects, (b) estimate variance components and associated G-coefficients representing the single site stability of MMN responses measured with different scoring methods separately at each of the eight NAPLS sites, and (c) compare such G-coefficients calculated using raw and z-scored MMN responses. Additional exploratory analyses are presented to demonstrate the Spearman-Brown prophecy (Brown, 1910;Spearman, 1910) in practice by averaging across MMN measured on separate occasions to capture more trait-like aspects of the MMN and relate it to subject traits.

| Participants
Participants were recruited at each of the eight NAPLS2 sites and all provided written, informed consent to participate in this IRBapproved study. EEG data were collected at baseline assessment from 241 HCs, and 167 (~70%) of these HCs completed at least one follow up EEG assessment. Additional demographic characteristics of these 167 subjects are presented in Table 1.
All HCs had at least one global assessment of functioning (GAF) as a part of study procedures (Endicott, Spitzer, Fleiss, & Cohen, 1976;Jones, Thornicroft, Coffey, & Dunn, 1995). The current GAF score nearest to the baseline EEG date (median time between GAF and EEG was 22 days, IQR: 7-34.61 days) was saved for correlation analyses. As an additional GAF metric, the mean current GAF score across all assessments (max = 5, one every 6 months) during the 24 month study period was saved. There were 18 HCs (~11%) who only had one GAF assessment, making their baseline GAF and mean GAF scores equivalent.

| MMN paradigm
All sites used similar hardware and presentation software (www. neurobs.com) to run the EEG experiment. Auditory stimuli were delivered via ER1-A Etymotic insert earphones and subjects responded with a Cedrus RB-830 button box. Auditory stimuli delivery consisted of 85% standard tones presented for 50 ms at 633 Hz, 5% duration (DUR) deviants presented for 100 ms at 633 Hz, 5% frequency (FRQ) deviants presented for 50 ms at 1000 Hz, and 5% double-deviants

| EEG data acquisition
EEG was digitized at 1024 Hz using 32-or 64-channel electrode caps (Biosemi, Amsterdam, The Netherlands), and the common 32 channels were used in subsequent steps. Additional electrodes were placed on the above and below the right eye, on the outer canthus of each eye, and on the mastoids. An offline average mastoid reference was initially applied to continuous EEG data prior to all preprocessing.
Blinks and eye movement artifacts were recorded by electrodes placed around the eyes and were subtracted from single trials using regression (Gratton, Coles, & Donchin, 1983). Following baseline correction (−50 to 0 ms), outlier electrodes were interpolated within single trial epochs based on previously established criteria (Nolan, Whelan, & Reilly, 2010).
A spherical spline interpolation (Delorme & Makeig, 2004) was applied to any channel that was determined to be a statistical outlier (|z| > 3) on one or more of four parameters, including variance to detect additive noise, median gradient to detect high-frequency activity, amplitude range to detect pop-offs, and deviation of the mean amplitude from the common average to detect electrical drift. Epochs were rejected, if they contained amplitudes greater than ±100 μV in any of these electrodes: AF3, AF4, F3, Fz, F4, FC1, FC2, FC5, FC6, C3, Cz, C4.

| ERP averaging and MMN measurement
ERP averages for all stimulus types were determined using a sorted averaging method, which has been shown to reduce noise in the MMN waveform by averaging over the subset of trials that optimizes the estimated signal to noise ratio for each subject (Rahne, von Specht, & Muhler, 2008 In Equation (1), y is the MMN score, a is the age, and e is the residual (i.e., difference between age-predicted and actual MMN score) for the ith subject. The model error, or specifically, root-meansquare error (RMSE), is calculated as: F I G U R E 1 A graphical representation of the design matrix used to predict mismatch negativity (MMN) responses is plotted. The first eight columns include an intercept term and seven site indicator variables, where white represents 1 and black 0. These columns capture fixed effects of site, while the ninth column is the age covariate, with grayscale age value representing each of the 241 healthy control participants' ages at baseline MMN assessment. This model is the common slope model where each site may have a different y-intercept, but a common age relationship estimated using data from all subjects and sites

Healthy Control Design Matrix
The RMSE summarizes the model's error across all subjects, and it can be used to calculate an age-corrected MMN z-score: In the multisite setting, one must consider laboratory site as between subjects, categorical variable. This increases the OLS model design matrix from two (intercept + age) to nine columns in NAPLS.  (Sakamoto & Kitagawa, 1987). False discovery rate (FDR) correction was applied separately to the two sets of F-tests, and Bonferroni correction was separately applied to families of tests limited to the electrodes (n = 32) for each measure and deviant type (p = 0.05/32 = 0.0015625). Finally, the number of uncorrected (p < .05) significant tests was listed for descriptive purposes.
Follow-up longitudinal MMN data were also z-scored using the baseline HC model and t-or F-tests were conducted to assess age and site effects, respectively, as additional measures of model fitness. This subset of 167 follow up data points could be considered a "hold-out" data set, and any site or age effects indicate that the z-scoring procedure suboptimally accounted for linear effects of age and fixed effects of site.

| Variance components and G-coefficients
The longitudinal HC data were re-purposed as a single facet (test occasion) G-study design to estimate variance components.
Such a design allows estimation of three variance components for any response using the data from the participants at a particular site. The variance components for Person (σ 2 p ), Occasion (σ 2 o ), and Person x Occasion plus Error (σ 2 po + e ) are estimated separately for each NAPLS laboratory site, as Site may represent another source of variance [see Roach et al. (2019)]. Once variance components are estimated, the G-coefficient, which provides a measure of generalizability or stability of the measured score in this longitudinal setup, can be calculated as in Equation (4): The NAPLS2 study design included EEG assessments at baseline, 12 month, and 24 month study time points. MMN scores from each session are treated separately, with particular emphasis on using baseline data to predict conversion to psychosis, meaning the best choice for n o is 1. Therefore, the G-coefficient is equal to the intraclass correlation (ICC) defined by Shrout and Fleiss (e.g., ICC (3,1) in (Shrout & Fleiss, 1979)) when n o = 1. Variance components were estimated using a restricted maximum likelihood approach in Matlab (Witkovský, 2012). Components were estimated separately and saved for the three deviant types (DBL, FRQ, DUR), 32 electrodes, and four MMN measurements (peak amplitude, mean around peak, mean in fixed window, and peak latency) for both MMN raw scores and z-scores. F I G U R E 4 Scatterplots depict the relationships between mismatch negativity (MMN) amplitude averaged across a fixed window at electrode Fz and participant years of age at testing for double (frequency plus duration; DBL, circles), frequency (FRQ, triangles), and duration (DUR, squares) deviants. Data are plotted separately for each site, and thick lines depict the common age relationship across sites based on regression models. Thin lines depict site-specific, nonlinear locally weighted predictions of MMN given age (Cleveland, Grosse, & Shyu, 1992), and there is no higher-order polynomial or other nonlinear pattern of fit that is consistent across sites G-coefficients. However, existing guidelines for determining clinical significance of ICCs suggest that the reliability coefficient can be qualitatively categorized as follows: ICC < 0.4 is poor, 0.4 ≤ ICC < 0.6 is fair, 0.6 ≤ ICC < 0.75 is good, and 0.75 ≤ ICC < 1 is excellent (Cicchetti & Sparrow, 1981). Therefore, G-coefficients were categorized using these 4 labels for descriptive purposes, as done previously .

| Exploratory correlations between MMN variables and trait variables
To capture more trait-like aspects of MMN, z-scores from 6 frontocentral electrodes (F3, Fz, F4, C3, Cz, C4) were averaged across the two test occasions separately for each deviant type and correlated with mean GAF or used to explore gender differences in MMN. As one method to demonstrate the enhanced reliability of averaged MMN z-scores, baseline MMN z-scores, and baseline GAF scores were also correlated. Similar to age regression models, site was a categorical covariate and heterogeneity of MMN-GAF relationships between sites were ruled out by first including a site*GAF interaction term, and the r 2 change F-test was used to determine improvement in model fit.
Given the exploratory nature of these correlations, parameter estimates, uncorrected p-values, as well as FDR-corrected p-values within this trait family of tests are reported.

| RESULTS
MMN ERP waveforms from electrode Fz are plotted in Figure 2. There is consistency between waveforms at each site and on each test occasion despite the long interval between tests and differences in site demographics. Descriptive statistics for trial numbers contributing to individual subject and test occasion ERPs are included in Table 2.

| Common slope linear regression models
There were two sets of F-tests to assess the appropriateness of common slope, site, and age regression models. In one set, a mean- F I G U R E 5 G-coefficients for the single-facet (test occasion) generalizability substudies calculated separately for each NAPLS geographic site for electrode Fz based on either raw (top) or standardized mismatch negativity (MMN) z-scores (bottom). Measurement approaches are plotted along the x-axis separately for double-deviant (DBL, circles), frequency-deviant (FRQ, triangles), and duration-deviant (DUR, squares) mismatch negativity. These include peak amplitude ("Peak": most negative peak between 90 and 290 ms in the MMN difference wave), mean amplitude ("Mean": ±10 ms around the peak), average amplitude in a fixed window ("Window": 90-170 ms for FRQ and DBL, 150-230 for DUR), and peak latency ("Latency"). Dashed lines indicate qualitative categorization of G-coefficients based on preexisting standards (Cicchetti & Sparrow, 1981)

T A B L E 3
Frequency of G-coefficients by NAPLS Site  Figure 4 for electrode Fz.
Tests of age relationships in the z-scored longitudinal follow-up MMN data indicated that the age effect was removed in this hold-out subsample, with only~5% (20/384) statistically significant effects at an uncorrected level (p < 0.05), consistent with what is expected by chance. None survived FDR correction. Tests of site effects in these data indicated~15% (58/384) were significant at an uncorrected level.
Only 3 site tests of MMN latency measures survived FDR correction.

| Variance components and G-coefficients
G-coefficients for each electrode, deviant type, measure, and NAPLS site are included in Table S1 for both raw MMN and z-scores. As can be seen in Figure 5, the G-coefficients for Fz are fair or better (G ≥ 0.4) in almost every MMN amplitude measure across NAPLS sites, but the latency G-coefficients are highly variable and poor in many cases.
Frequencies of poor, fair, good, and excellent reliability categorization of all G-coefficients are presented in Table 3 separated by NAPLS site. The table demonstrates that the majority (~60%) of G-coefficients were fair or better, including many (~27%) scores with excellent generalizability. G-coefficients based on z-scores were nearly equivalent to those based on raw scores (average difference in G-coefficients = 0.0044), consistent with relatively small proportions of MMN raw score variance being accounted for by age and site. Site-specific relationships between mean MMN amplitude in a fixed window on the first and second test occasions are plotted along with corresponding deviant-specific scatter plots in Figure 6 for electrode Fz.

| Exploratory correlations between MMN variables and trait variables
Parameter estimates along with test statistics for all trait-like MMN models are presented in Table 4. There were no significant site*GAF  (Table 4). In the gender models, there was neither evidence of a site*gender interaction effect for any deviant type (all ps > 0.487), nor evidence of a gender difference between males and females in the reduced models.

| DISCUSSION
One goal of this study was to present a site and age modeling strategy to create regression models to produce standardized site-and age- Several alternatives to the z-score approach for removing site-and age-related confounds in clinical studies are potentially problematic. One alternative is to ignore site-and age-related variation.
A second approach would be to eliminate subjects from certain sites in order to match groups on age at each site. In studies of rare outcome events or patients, which is one of the motivations of a multisite study like NAPLS, eliminating subjects is disadvantageous. A third approach is to conduct ANCOVA with site and age as a covariates. The problem with an age factor in ANCOVA is that MMN-age relationships are derived from a pooled estimate of aging effects from all of the groups being compared, including the CHR. On theoretical grounds, it is reasonable to hypothesize that physiological measures like MMN in the psychosis prodrome may have abnormal age trajectories, reflecting abnormal brain maturation, and other pathogenic processes operating during the transition to psychosis or disease-related progressive brain changes occurring after illness onset (Kiang, Braff, Sprock, & Light, 2009;Light et al., 2015;Todd et al., 2008). Accordingly, we believe ANCOVA models are inappropriate because of their potential to remove disease-related aging effects along with normal aging effects within the study sample.
Two previous studies reported excellent MMN reliability (Fz ICC > 0.8) using a long duration deviant similar to this study and a window measurement (135-205 ms) from nose-referenced data Light & Braff, 2005). These reliability coefficients were based on either 10 patients with schizophrenia (Light & Braff, 2005), 168 patients with schizophrenia, or 58 healthy subjects   Lew et al. (2007)].
The Spearman-Brown prophecy formula indicates that reliability of a score increases as test length or the number of items averaged to summarize a subject's score increases (Brown, 1910;Spearman, 1910 Light et al., 2015), it is feasible to combine data from multiple, appropriately controlled and calibrated, research laboratory sites to study MMN.

CONFLICT OF INTEREST
The authors declare no potential conflict of interest.