{'fraction_non_alphanumeric': 0.040688820336600415, 'fraction_numerical': 0.02077394665937756, 'mean_word_length': 5.312546216416071, 'pattern_counts': {'":': 0, '<': 6, '<?xml version=': 0, '>': 14, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Frequency of Abnormally Low Neuropsychological Scores in Post-COVID-19 Syndrome: the Geneva COVID-COG Cohort

Abstract Objective Several studies have reported poor long-term neuropsychological performances in patients following severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection, but none has yet considered the effect of administering multiple intercorrelated neuropsychological tests and assessed the frequency of cognitive deficits in a normative population. Our aim was therefore to assess the presence of cumulative neuropsychological deficits in an actual post-coronavirus disease of 2019 (COVID-19) comparison group versus one simulated using Monte-Carlo methods. Method Validated neuropsychological Monte-Carlo simulation methods were applied to scores from a battery of neuropsychological tests (memory, executive, attentional, perceptual, logical reasoning, language, and ideomotor praxis) administered to 121 patients who had had mild, moderate, or severe COVID-19 (mean age: 56.70 years; 32% women), 222 ± 43 days post-infection. The cumulative percentages of the three severity subgroups were compared with the results of a false discovery rate-corrected probability analysis based on normative data. Results The cumulative percentages of deficits in memory and executive functions among the severe and moderate patients were significantly higher than those estimated for the normative population. Moderate patients also had significantly more deficits in perception and logical reasoning. In contrast, the mild group did not have significantly more cumulative deficits. Conclusions Moderate and severe forms of COVID-19 cause greater long-term neuropsychological deficits than those that would be found in a normative population, reinforcing the hypothesis of long-term effects of SARS-CoV-2 on cognitive function, independent of the severity of the initial infection.

Little is known about the frequency of long-term neuropsychological deficits following severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection. To date, most studies have assessed the frequency of neuropsychological deficits using global cognitive efficiency tests such as the Montreal Cognitive Assessment (MoCA), Mini-Mental State Examination (MMSE), and Telephone Interview for Cognitive Status (for review see Daroische, Hemminghyth, Eilertsen, Breitve, & Chwiszczuk, 2021). Few studies have assessed this frequency using neuropsychological tests that probe specific cognitive functions (Almeria, Cejudo, Sotoca, Deus, & Krupinski, 2020;Jaywant et al., 2021;Krishnan, Miller, Reiter, & Bonner-Jackson, 2022). Adopting a conservative methodology (5th percentile cutoff), Almeria and colleagues (2020) reported impaired (5.70%-11.40%) scores on all assessed functions 10-35 days post-discharge on average after a SARS-CoV-2 infection. Group comparisons on raw scores revealed significantly lower scores for patients who had been in an intensive care unit (ICU) than for patients who had been hospitalized but not in ICU, as well as for patients who had received oxygen therapy, versus those who had not (Almeria et al., 2020). When Jaywant and colleagues (2021) used a combination of conservative and nonconservative (borderline scores below 16th percentile) methodologies to assess the frequency of memory, executive, and attention deficits in 57 patients 43.2 days after initial admission, they observed deficits in all functions (46% of the sample in the case of divided attention). Almeria and colleagues (2020) and Jaywant and colleagues (2021) highlighted the short-term frequency of deficits (<3-month post-infection), whereas Krishnan and colleagues (2022) assessed 20 mild-to-moderate patients 5.5 months post-infection and highlighted a frequency of deficits ranging from 5% to 52% for memory and executive function tests, suggesting a potentially high frequency of long-term neuropsychological deficits in patients with mild infection. However, these studies had two main methodological biases. First, there was a lack of analysis to determine whether the neuropsychological deficits were greater than would be expected in a normative population, depending on the number of tests performed (the higher the number of neuropsychological tests, the greater the probability of observing a subnormal neuropsychological score, even in a normative population; Crawford, Garthwaite, & Gault, 2007). Over the past two decades, studies among populations of neurotypical participants without any cognitive complaints have shown that the application of frequency calculation methodologies, be they conservative or nonconservative, can lead to the discovery of a high percentage of deficits in the normative population (for a review, see Binder, Iverson, & Brooks, 2009). Furthermore, the risk of intercollinearity (i.e., two or more variables being completely correlated) of the neuropsychological tests was not considered in these studies, even though it could potentially indicate an accumulation of deficits in the various cognitive functions being assessed. Running simulation analyses of neuropsychological data, as proposed by Crawford and colleagues (2007), would therefore allow authors to determine whether a sample's accumulated deficits are higher than would be expected in a normative population or in a clinical subgroup of patients. Second, a lack of analysis with dedicated tools for evaluating the presence of symptoms to be considered with care and the validity of responses may have led to a superficially high frequency of neuropsychological deficits (Binder et al., 2009).
In this context, the present study had three main objectives: (a) investigate whether SARS-CoV-2 infection causes long-term (6-9 months after acute phase) neuropsychological deficits, controlling for the validity of symptoms; (b) identify the nature of the affected cognitive domains using a comprehensive neuropsychological assessment to probe multiple cognitive domains; and (c) determine whether the cumulative percentage of neuropsychological deficits as a function of severity of acute infection is higher than would be expected in a normative population, by considering possible intercorrelations between the neuropsychological tests used. Patients were divided into three subgroups according to the respiratory severity (severe, moderate, and mild) of their disease in the acute phase, and their cumulative percentages were then compared with scores estimated for a normative population, based on normative data. Previous studies had highlighted impaired neuropsychological performances (raw scores; Almeria et al., 2020;Jaywant et al., 2021) and a higher frequency of deficits in mild-to-moderate patients (Krishnan et al., 2022). We hypothesized that coronavirus disease of 2019 (COVID-19) causes long-term neuropsychological deficits that continue to affect patients' functioning 6-9 months post-infection. More specifically, we expected to observe a higher cumulative percentage (frequency) of abnormally low scores on memory and executive functions among patients who had moderate-to-severe acute infections (hospitalized) than that estimated for the normative population using the Monte-Carlo simulation method developed by Crawford and colleagues (2007), validated by Schretlen, Testa, Winicki, Pearlson, and Gordon (2008), and adapted for our study.

Participants
Of all the patients who had an infection confirmed by positive polymerase chain reaction (PCR) results from a nasopharyngeal swab and/or by positive serological results between March 2020 and May 2021 at Geneva University Hospitals (HUG) (∼4,000 patients), ∼300 patients met the inclusion criteria (see below), and of these a total of 121 agreed to participate in the study 222.46 ± 42.93 days after their infection. According to the severity of their infection in the acute phase, these patients were divided into three subgroups: mild (not hospitalized; n = 49), moderate (hospitalization without mechanical ventilation; n = 48), or severe (ICU and mechanical ventilation; n = 24; see Table 1). During the inclusion phase, the mild and moderate groups were matched to patients in the severe group so that the three groups would be comparable on median age, sociocultural level, language (all were French-speaking Swiss citizens or residents of the French part of Switzerland), and clinical variables (see Table 1). Participants were recruited via the CoviCare program , which follows patients with post-COVID symptoms in Geneva, Switzerland (MN, OB, and IG), as well as from the registers of another study (LB; Benzakour et al., 2021). For each patient, we carried out a medical file review, followed by a telephone call inviting the patient to take part in the study if all the eligibility criteria were met. Exclusion criteria were a history of neurological issues, psychiatric disorders (two of the included participants had had an episode of depression >10 years before their SARS-CoV-2 infection), cancer and neurodevelopmental pathologies, pregnancy, and age above 80 years. The frequency of the different clinical symptoms experienced by patients during the acute phase is indicated in Table 2.

General Procedure and Ethics
After being given a full description of the study, participants provided their written informed consent. The study was conducted in accordance with the Declaration of Helsinki, and the study protocol was approved by the cantonal ethics committee of Geneva (CER-02186).

Neuropsychological Assessment
A comprehensive neuropsychological battery solely containing tests validated in a French-speaking population was administered in French (all patients were fluent in written and spoken French). This battery included a series of tests and questionnaires that assessed most of the domains of cognition (for testing details, see Table 3). The tests were administered by certified clinical psychologists (mean duration: ∼180 min). A neurological assessment was carried out by two board-certified neurologists (FA and GA; see Voruz et al., 2022a).

Measure of Symptom Validity
We used the BRIEF-A (Abeare et al., 2021) and WAIS-IV digit-span measures (Kanser, Rapport, Hanks, & Patrick, 2021) to measure the validity of the patients' performance, the validity (i.e., congruence) of their symptoms, and the presence of any symptoms to be considered with care. The default validity check of the BRIEF-A (negativity scale ≥ 6) yielded good-to-excellent   results for all participants, as did our use of the cutoff purposed by Abeare and colleagues (2021) (failure on ≥4 of the individual scales). These good-to-excellent scores on the BRIEF-A, were partially confirmed by the validity measured by the WAIS-IV Digit Span Forward and Backward with a threshold of ≥5 measured in a healthy control population and which simulated (Kanser et al., 2021). In this case, only one participant had a score above the threshold on the Digit Span Forward, and two patients on the Digit Span Backward. Although they performed well on the BRIEF-A, these patients showed other attentional deficits, as measured with the Test for Attentional Performance Version 2.1 (Zimmermann & Fimm, 2007), suggesting genuine impairment.

Statistical Analyses
For each neuropsychological test, we first compared patients' performances with normative data collected from a reference sample for the validated neuropsychological tools. Raw scores were converted to standardized scores and adjusted for age, education, and gender, using published normative data and in accordance with the guidelines of the Swiss Association of Neuropsychology (Frei et al., 2016;Heaton, Grant, & Matthews, 1991), allowing us to quantify the frequency of each type of impairment.
To determine whether the deficits we observed in our patient cohort were greater than those that would be expected in a normative population, and to compensate for the number of tests, we calculated the frequency of low scores using the Monte-Carlo simulation program (i.e., estimated base rates), with a framework developed by Crawford and colleagues (2007) and validated in the neuropsychology field. This method estimates the baseline rates of low scores according to test intercorrelations (in our case, for the whole sample of patients with SARS-CoV-2). It allowed us to estimate, as recommended by Crawford and colleagues (2007), the percentage of the normative population who would exhibit one or more, two or more, three or more, or four or more abnormally low scores, applying a conservative threshold (< 5th percentile). We carried out six steps. (a) We pooled the scores according to the cognitive functions they were supposed to assess, based on theoretical models and/or test batteries: episodic memory model (Schacter & Tulving, 1994) and The Memory NEo-Structural Inter-Systemic model (MNESIS) model (Eustache, Viard, & Desgranges, 2016) for memory; latent model of executive functions (Miyake et al., 2000) and GREFEX battery (Roussel & Godefroy, 2008)  We calculated the cumulative percentages of patients in each group (total sample; mild, moderate, and severe subgroups) with at least one lower test score. (f) We compared these cumulative percentages with the estimated scores of the normative population (obtained at Step 3 using binomial distribution probability analyses, as suggested by Crawford and colleagues (2007), with the estimated baseline rate used to specify p sample: (i) estimate versus mild group; (ii) estimate versus moderate group; and (iii) estimate versus severe group. Finally, we applied a Benjamini-Hochberg false discovery rate (FDR) correction with a p value set at .050 for each group comparison (estimate vs. total ere) on each of the functions we assessed (memory, executive, attentional, perceptual, logical reasoning, language, and ideomotor praxis).

Perceptual Functions
The total sample and the normative population only differed significantly on four or more abnormally low scores [total sample has 0.82% more abnormally low scores than the estimated score for the normative population (+ 0.82%, p < .001)]. All other comparisons were nonsignificant (ps > .05; Table 4).
For the moderate subgroup, significant differences were found with the normative population on two or more (+ 8.09%, p < .001), three or more (+ 1.86%, p < .001), and four and more (+ 2.07%, p < .001) abnormally low scores. No other significant differences were found (all ps > .05).
For the mild and severe subgroups, all cumulative percentages were nonsignificant, compared with the normative population (all ps > .05).

Language and Ideomotor Praxis
Analysis revealed no significant differences between the normative population and either the total sample or the mild, moderate, and severe subgroups (all ps > .05; Table 4).

100% 0%
Note. Mild: patients not hospitalized for SARS-CoV-2 infection; Moderate: patients hospitalized without mechanical ventilation for SARS-CoV-2 infection; and Severe: patients hospitalized in intensive care with mechanical ventilation for SARS-CoV-2 infection. Note. * * Cumulative percentages of patients significantly above the estimated percentage for the normative population after FDR correction.
The mild subgroup did not differ significantly from the normative population on any of the cumulative percentages (all ps > .05).
The severe subgroup differed significantly from the normative population on one or more (+ 18.51%, p = .018) and two or more (+ 16.46%, p = .008) abnormally low scores, but the three other differences did not survive FDR correction (all ps > .05).

Attentional Functions
The total sample differed significantly from the normative population on three or more abnormally low scores (+ 3.75%, p = .015), but all other cumulative percentage comparisons were nonsignificant (all ps > .05; Table 4).
For the mild, moderate, and severe subgroups, no significant differences were found after FDR correction (all ps > .05).

Memory Functions
The total sample differed significantly from the normative population on two or more (+ 7.72%, p = .003), four or more (+ 1.96%, p = .001), and five or more abnormally low scores (+ 2.86%, p = .002). Comparisons on one or more, three or more, and six or more abnormally low scores were nonsignificant after FDR correction (all ps > .05; Table 4 and Fig. 1).
No significant differences were found between the mild subgroup and the normative population on any of the cumulative percentages (all ps > .050).
The severe subgroup differed significantly from the normative population after FDR correction on one or more (+ 25.08%, p = .002) and two or more (+ 15.36%, p = .017) abnormally low scores, but comparisons for three or more, and five or more abnormally low scores did not survive FDR correction. All other comparisons were nonsignificant (all ps > .050).

Logical Reasoning
The total sample and mild and severe subgroups did not differ significantly from the normative population in either comparison (both ps > .05). For the moderate subgroup, a significant difference was found on two or more abnormally low scores (+ 0.65%, p = .032), but not on one or more abnormally low scores (p > .05; Table 3).

Discussion
To date, it has been difficult to determine the frequency of neuropsychological deficits in post-COVID-19 condition. Although research has highlighted a high frequency of neuropsychological deficits in various cognitive functions (Almeria et al., 2020;Jaywant et al., 2021;Krishnan et al., 2022), authors have not envisaged the possibility that this frequency could be comparable with that observed in a normative population (Binder et al., 2009). The present study was designed to address this issue, by comparing the cumulative frequency of neuropsychological deficits in patients who had had COVID-19 (mild, moderate, or severe) at 222.46 ± 42.93 days post-infection with estimated percentages for a normative population, using a robust and validated neuropsychological simulation methodology. As expected, the moderate and severe subgroups had higher cumulative percentages of memory and executive function deficits than the normative population. Moreover, our analysis suggested a greater frequency of perceptual and logical reasoning deficits in moderate (but not mild or severe) patients, compared with estimates for the normative population.
For the first time in a population of patients with no clinical history prior to the infection, our analyses showed that the frequency of neuropsychological deficits 6-9 months following SARS-CoV-2 infection was higher than would be observed in a normative population, suggesting that SARS-CoV-2 infection has long-term neuropsychological consequences. For the methodological reasons set out above, it proved difficult to compare our results with those of other studies that have assessed the frequency of neuropsychological deficits following SARS-CoV-2 infection. We were nevertheless able to do so to some extent in order to interpret our results. In the present study, calculating cumulative percentages of patients with neuropsychological deficits freed us from the previously mentioned methodological limitation and allowed us to compare these percentages with those estimated for a normative population, as recommended in the methodological literature on the assessment of the frequency of deficits in neuropsychology (Crawford et al., 2007). Moreover, the neuropsychological deficits were not confined to the severe subgroup (i.e., patients who required mechanical ventilation), as they were also present in the moderate subgroup, and even appeared to be of greater severity (major perceptual and logical reasoning deficits), confirming the observations of Alemanno and colleagues (2021), who highlighted significantly greater neuropsychological deficits on the MoCA and MMSE screening batteries in moderate patients (hospitalized without mechanical ventilation) than in severe patients (hospitalized with mechanical ventilation). This suggests that post-ICU effects are not the sole cause of long-term neuropsychological deficits (Sakusic et al., 2018), given the high cumulative percentages of memory, executive, perceptual, and logical reasoning deficits in the moderate subgroup. Moreover, this highlights central nervous system involvement, either directly via SARS-CoV-2 neuroinvasive mechanisms, or indirectly via an immunological and/or inflammatory response to SARS-CoV-2 (Bougakov, Podell, & Goldberg, 2021), which is supported by PET-FDG (Guedj et al., 2021), functional connectivity (Voruz et al., 2022b), and structural (Douaud et al., 2022) neuroimaging. Another hypothesis, which could explain the greater number of cumulative deficits in moderate patients, concerns the level of hypoxemia, for according to the literature, this is a predictor of cognitive deficits (Herridge et al., 2016;Mikkelsen et al., 2012). Thus, patients who were not in intensive care and intubated may have experienced more prolonged hypoxemia, which would explain these greater neuropsychological deficits. Our results do not allow us to state exactly how the central nervous system is affected in COVID-19, but do suggest that the substantial variance in longterm neuropsychological deficits in COVID-19 is attributable to the infection, and not to previous medical history, treatment, or post-ICU effects. In contrast, our analyses failed to reveal higher cumulative percentages of deficits for the mild subgroup than for the normative population, despite a trend toward significance (nonsignificant after FDR correction) for attention. The cumulative percentages for the other functions were significantly lower, in contrast to previous observations among mild patients (Krishnan et al., 2022). Nevertheless, this does not exclude the possibility that some patients may have experienced long-term effects such as attentional or concentration problems, with an impact on their daily lives. Some patients may well have had neuropsychiatric symptoms, which are described in the literature as being very frequent (for review see Mazza et al., 2020;Schou, Joca, Wegener, & Bay-Richter, 2021). These symptoms may sporadically influence neuropsychological performance and patients' daily life, without reaching large cumulative percentages.

Limitations
The present study had five limitations. First, our subgroup of mild patients may have had a moderate or severe infection, but not have gone to hospital. This may have been the effect of sociocultural status and gender, and may have resulted in long-term cognitive deficits, owing to poor disease management. Second, although our study had strict exclusion criteria, the premorbid features (e.g., respiratory, cardiac, or endocrine pathologies) that we documented in some of our patients may have induced cognitive deficits. However, few of our patients reported comorbidities, and according to the literature, most of these comorbidities are not known to induce cognitive deficits, if treated (e.g., four of our patients reported asthma). Third, it is currently impossible for performances on a neuropsychological test to be determined by a single cognitive function. All the evidence suggests that performances are cognitively multidetermined, thereby limiting their interpretation (Bilder & Reise, 2019). Nevertheless, our pooling to measure neuropsychological subfunctions was based on validated cognitive models and validated test batteries. Fourth, a control group could have been included, but because of the high rate of infection in Geneva across 2020 and 2021 and our exclusion criteria (absence of relevant clinical history), plus the necessity of antibody testing, there was a high risk of including patients who had had a mild disease, which would have biased the study. Fifth, our analysis was an exploratory one with 121 participants. This may have influenced the generalizability of the results, and there may have been an overall tendency for the level of abnormality in each separate comparison to be overestimated (Crawford & Garthwaite, 2005). We sought to overcome this difficulty by incorporating a correction for multivariate comparisons (i.e., FDR correction), which allowed us to partially correct this tendency.
Despite these limitations, our study comparing a cohort of patients with no relevant clinical history, 6-9 months after COVID-19 infection, with a normative population created using a statistical simulation methodology validated in neuropsychology, yielded evidence of major cumulative deficits in neuropsychological functions such as memory and executive functions. These were found not only in patients who had been in ICU, but also in those who had had moderate disease, suggesting a potential direct or indirect impact of the infection on cognitive functions.

Conclusion
Our results highlighted a higher frequency of neuropsychological deficits (mainly affecting memory and executive functions) among moderate and severe patients with post-COVID-19 condition than would be expected in a normative population. They highlight the importance of considering these symptoms in the management of patients with post-COVID-19 condition.

Funding
The present research was supported by Swiss National Science Foundation (SNSF) grants to J. A. Péron (PI) and Frédéric Assal (Co-PI) within the framework of the COVID-19 National Research Program (NRP 78; Grant no. 407840_198438, RNP 78). The funders had no role in data collection, discussion of content, preparation of the manuscript, or decision to publish. We would like to thank the patients for contributing their time to this study.