{'fraction_non_alphanumeric': 0.03815024359578815, 'fraction_numerical': 0.030174446016030174, 'mean_word_length': 5.100910834132311, 'pattern_counts': {'":': 0, '<': 2, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
6-month neurological and psychiatric outcomes in 236 379 survivors of COVID-19: a retrospective cohort study using electronic health records

Summary Background Neurological and psychiatric sequelae of COVID-19 have been reported, but more data are needed to adequately assess the effects of COVID-19 on brain health. We aimed to provide robust estimates of incidence rates and relative risks of neurological and psychiatric diagnoses in patients in the 6 months following a COVID-19 diagnosis. Methods For this retrospective cohort study and time-to-event analysis, we used data obtained from the TriNetX electronic health records network (with over 81 million patients). Our primary cohort comprised patients who had a COVID-19 diagnosis; one matched control cohort included patients diagnosed with influenza, and the other matched control cohort included patients diagnosed with any respiratory tract infection including influenza in the same period. Patients with a diagnosis of COVID-19 or a positive test for SARS-CoV-2 were excluded from the control cohorts. All cohorts included patients older than 10 years who had an index event on or after Jan 20, 2020, and who were still alive on Dec 13, 2020. We estimated the incidence of 14 neurological and psychiatric outcomes in the 6 months after a confirmed diagnosis of COVID-19: intracranial haemorrhage; ischaemic stroke; parkinsonism; Guillain-Barré syndrome; nerve, nerve root, and plexus disorders; myoneural junction and muscle disease; encephalitis; dementia; psychotic, mood, and anxiety disorders (grouped and separately); substance use disorder; and insomnia. Using a Cox model, we compared incidences with those in propensity score-matched cohorts of patients with influenza or other respiratory tract infections. We investigated how these estimates were affected by COVID-19 severity, as proxied by hospitalisation, intensive therapy unit (ITU) admission, and encephalopathy (delirium and related disorders). We assessed the robustness of the differences in outcomes between cohorts by repeating the analysis in different scenarios. To provide benchmarking for the incidence and risk of neurological and psychiatric sequelae, we compared our primary cohort with four cohorts of patients diagnosed in the same period with additional index events: skin infection, urolithiasis, fracture of a large bone, and pulmonary embolism. Findings Among 236 379 patients diagnosed with COVID-19, the estimated incidence of a neurological or psychiatric diagnosis in the following 6 months was 33·62% (95% CI 33·17–34·07), with 12·84% (12·36–13·33) receiving their first such diagnosis. For patients who had been admitted to an ITU, the estimated incidence of a diagnosis was 46·42% (44·78–48·09) and for a first diagnosis was 25·79% (23·50–28·25). Regarding individual diagnoses of the study outcomes, the whole COVID-19 cohort had estimated incidences of 0·56% (0·50–0·63) for intracranial haemorrhage, 2·10% (1·97–2·23) for ischaemic stroke, 0·11% (0·08–0·14) for parkinsonism, 0·67% (0·59–0·75) for dementia, 17·39% (17·04–17·74) for anxiety disorder, and 1·40% (1·30–1·51) for psychotic disorder, among others. In the group with ITU admission, estimated incidences were 2·66% (2·24–3·16) for intracranial haemorrhage, 6·92% (6·17–7·76) for ischaemic stroke, 0·26% (0·15–0·45) for parkinsonism, 1·74% (1·31–2·30) for dementia, 19·15% (17·90–20·48) for anxiety disorder, and 2·77% (2·31–3·33) for psychotic disorder. Most diagnostic categories were more common in patients who had COVID-19 than in those who had influenza (hazard ratio [HR] 1·44, 95% CI 1·40–1·47, for any diagnosis; 1·78, 1·68–1·89, for any first diagnosis) and those who had other respiratory tract infections (1·16, 1·14–1·17, for any diagnosis; 1·32, 1·27–1·36, for any first diagnosis). As with incidences, HRs were higher in patients who had more severe COVID-19 (eg, those admitted to ITU compared with those who were not: 1·58, 1·50–1·67, for any diagnosis; 2·87, 2·45–3·35, for any first diagnosis). Results were robust to various sensitivity analyses and benchmarking against the four additional index health events. Interpretation Our study provides evidence for substantial neurological and psychiatric morbidity in the 6 months after COVID-19 infection. Risks were greatest in, but not limited to, patients who had severe COVID-19. This information could help in service planning and identification of research priorities. Complementary study designs, including prospective cohorts, are needed to corroborate and explain these findings. Funding National Institute for Health Research (NIHR) Oxford Health Biomedical Research Centre.

time is monitored to ensure data validity. At least one non-demographic fact for each patient is required for them to be counted in the dataset. Patient records with only demographics information are discarded.
The software also undergoes quality control. The engineers testing the software are independent from the engineers developing it. Each test code is checked by two independent testing engineers. Each piece of software is tested extensively against a range of synthetic data (i.e. generated for the purpose of testing) for which the expected output is established independently. If the software fails to return this output, then the software is deemed to have failed the test and is examined and modified accordingly. For statistical software (including that used for propensity score matching, for Kaplan-Meier analysis, etc), an additional quality control step is implemented. Two independent codes are written in two different programming languages (typically R and python) and the statistical results are compared. If discrepancies are identified, then the codes are deemed to have failed the test and are examined and modified accordingly. All the code is reviewed independently by another engineer.
The test strategy follows three levels of granularity: 1. Unit tests: These test specific blocks, or units, of code that perform specific actions (e.g. querying the database). 2. Integration tests: These ensure that different components are working together correctly. 3. End-to-end tests: These tests run the entire system and check the final output.

Some comments on advantages and disadvantages of EHR data
The advantage of EHR data, like those in TriNetX, over insurance claim data is that both insured and uninsured patients are included. An advantage of EHR data over survey data is that they represent the diagnostic rates in the population presenting to healthcare facilities. This provides an accurate account of the burden of specific diagnoses on healthcare systems. The downside of relying on diagnoses is that they obviously do not account for undiagnosed patients who might be suffering from the illness but did not seek medical attention (or in whom the diagnosis was missed). A general limitation of EHR data is that a patient may be seen in different HCOs for different parts of their care and if one HCO is not part of the federated network then part of their medical records may not be available. Using a network of HCOs (rather than a single HCO) limits this possibility but does not fully remove it. Finally, historical data before the start of EHRs (or the addition of an HCO to the network) may be incomplete.

Definition of cohorts
The two control cohorts used consisted of patients with a diagnosis of influenza and patients with another respiratory tract infection. Specifically, patients with influenza were those who had any of the following diagnoses: -J09: Influenza due to certain identified influenza viruses -J10: Influenza due to other identified influenza virus -J11: Influenza due to unidentified influenza virus.
Patients with another respiratory tract infection were those with any of the following diagnosis: -J00: Acute nasopharyngitis -J01: Acute sinusitis -J02: Acute pharyngitis -J03: Acute tonsillitis -J04: Acute laryngitis and tracheitis -J05: Acute obstructive laryngitis [croup] and epiglottitis -J06: Acute upper respiratory infections of multiple and unspecified sites -J09: Influenza due to certain identified influenza viruses -J10: Influenza due to other identified influenza virus -J11: Influenza due to unidentified influenza virus -J12: Viral pneumonia, not elsewhere classified -J13: Pneumonia due to Streptococcus pneumoniae -J14: Pneumonia due to Hemophilus influenzae -J15: Bacterial pneumonia, not elsewhere classified -J16: Pneumonia due to other infectious organisms, not elsewhere classified -J17: Pneumonia in diseases classified elsewhere -J18: Pneumonia, unspecified organism -J20: Acute bronchitis -J21: Acute bronchiolitis -J22: Unspecified acute lower respiratory infection Because some patients with the control index event might have had COVID-19 at a different point in time, we excluded from the control cohorts all those who had COVID-19 at any point in time. To avoid any contamination between cohorts, COVID-19 as an exclusion criterion was defined in the broader sense to be all patients with a confirmed diagnosis of COVID-19 (ICD-10 code U07.1) but also patients with an unconfirmed COVID-19 diagnosis (U07.2), a recorded positive PCR test for COVID-19, or any of the following recorded on or after January 20, 2020: Pneumonia due to SARS-associated coronavirus (J12.81), Other coronavirus as the cause of disease classified elsewhere (B97.29), or Coronavirus infection unspecified (B34.2). Inclusion of the latter three diagnostic codes captures patients who receive a COVID-19 diagnosis in the early stage of the pandemic when the ICD code for COVID-19 (U07) was not yet defined. Specifically, the following codes were excluded from the control cohorts if they occurred on or after January 20, 2020: The duration of follow-up of the patients depended on when they had the index event. Patients who had the index events more than 6 months before the date of the analysis (December 13, 2020) had 6 months of followup. The other patients were followed up until December 13, 2020. The Kaplan-Meier estimator accommodates differences in duration of follow-up by means of censoring.

Definition of covariates
To reduce the effect of confounding on associations between a diagnosis of COVID-19 and a subsequent neurologic or psychiatric diagnosis, cohorts were matched for established or suspected risk factors for COVID-19 2-5 and for established risk factors for COVID-19 death 6 (taken to be risk factors of a more severe COVID-19 illness). The following confounding factors were therefore included (with ICD-10/CDC codes in brackets): 1) Age at the time of diagnosis 2) Sex coded as female, male, or other.

7)
Hypertension encoded as 2 dichotomous and 2 categorical variables: Hypertensive diseases (I10-I16), the now deprecated version that was used until 2018 Hypertension diseases (I10-I15), measurements of systolic blood pressure (categorised into < 140mmHg, 140-160mmHg, and ≥ 160mmHg), and diastolic blood pressure (categorised into < 90mmHg, 90-100mmHg, and ≥ 100mmHg). The blood pressure categories correspond to the absence of hypertension, stage 1 hypertension, and stage 2 (and over) hypertension as per the NICE guidelines. 8) Diabetes mellitus encoded as 2 dichotomous variables: Type 1 diabetes mellitus (E10) and Type 2 diabetes mellitus (E11) 9) Chronic lower respiratory diseases encoded by each sub-category of the corresponding ICD-10 group: Bronchitis, not specified as acute or chronic (J40), Simple and mucopurulent chronic bronchitis (J41), Unspecified chronic bronchitis (J42), Emphysema (J43), Other chronic obstructive pulmonary disease (J44), Asthma (J45), Bronchiectasis (J47) 10) Nicotine dependence encoded as the corresponding ICD-10 diagnosis (F17.2) 11) Substance use disorder encoded as the ICD-10 code for mental and behavioral disorders due to psychoactive substance use (F10-F19) 12) Heart diseases encoded as 2 categorical variables: Ischaemic heart disease (I20-I25) and Other forms of heart disease (I30-I52) 13) Chronic kidney disease encoded as 2 dichotomous variables: Chronic kidney disease (N18) and Hypertensive chronic kidney disease (I12) 14) Chronic liver disease encoded as 8  Lupus encoded as a dichotomous variable corresponding ICD-10 code (M32) 21) Psoriasis encoded as a dichotomous variable corresponding ICD-10 code (L40) 22) Disorders involving an immune mechanism encoded as a dichotomous variable "Certain disorders involving the immune mechanism" (D80-D89) Each individual code was considered a confounding factor in and of itself so that matching was achieved for each of them individually. For instance, matching was achieved for each subcategory (and not just for the whole category) of chronic lower respiratory diseases. For variables representing diagnoses and socioeconomic deprivation, an individual was considered positive if the diagnostic was recorded at least once in their health record before the index event. For categorical variables representing measurements (i.e. BMI and blood pressures), all available measurements for all individuals were used and propensity score matching sought to define cohorts with similar numbers of measurements falling into each category.

Definition of outcomes
All outcomes were defined as a diagnosis recorded in the patient's electronic health record between 1 day and 180 days after the index event. As mentioned in the manuscript, for chronic illnesses, only first diagnoses were counted (i.e. patients with the diagnosis before the index event were excluded from the survival analysis). For diagnoses that can recur or relapse, we separately estimated the incidence of first diagnosis and the incidence of any diagnoses. For all other diagnoses, we estimated the incidence of any diagnoses.

Details on secondary analyses
Encephalopathy Encephalopathy was defined as the presence of any of the following diagnostic codes between 4 days before and 2 weeks after the COVID-19 diagnosis: -Other and unspecified encephalopathy (G93.4), -Delirium (F05), -Other mental disorders due to known physiological condition (F06), -Personality and behavioral disorders due to known physiological condition (F07), -Disorientation (R41.0), -Somnolence (R40.0), -Stupor (R40.1). The combination of these diagnostic codes aimed to capture various clinical presentations of encephalopathy which can all represent a change from baseline cognitive status 7 . Furthermore, they account for semantic differences across disciplines. Our choice of the term 'encephalopathy' to characterise this cohort reflects both the fact that other codes are clinical manifestations of encephalopathy, and the fact that encephalopathy (G93.4) was the most prevalent code used in this cohort (present in 66.0% of patients).

Benchmarking against four additional index health events
To provide benchmarks for the incidence of neurologic and psychiatric sequelae of health events, we compared these incidences between the COVID-19 cohort and 4 other matched cohorts of patients defined by another index event (similarly to our previous study 1 ): -Skin infection: ICD-10 codes L00-L08 ('Infection of the skin and subcutaneous tissue'). This health event was selected as a common infective event that does not affect the respiratory tract. -Urolithiasis: ICD-10 codes N20-N23. This health event was selected as a common surgical presentation. -Fracture of a large bone: ICD-10 codes S32 ('Fracture of lumbar spine and pelvis'), S42 ('Fracture of shoulder and upper arm'), S52 ('Fracture of forearm'), S72 ('Fracture of femur'), S82 ('Fracture of lower leg including ankle'). This health event was selected as it is largely independent of prior medical conditions, and almost always leads to presentation for treatment. -Pulmonary embolism: ICD-10 code I26. This health event was selected since, it is an acute and lifethreatening event and so may have similarities to COVID-19 in terms of its psychological impact.
This analysis was achieved using the same survival analysis as in the primary analysis and by only altering the index event of the control cohort. These additional control index events were selected as they represent a broad range of common acute presentations with a range of anticipated neurologic and psychiatric sequelae.

Additional subgroup analyses
We assessed the robustness of the differences in outcomes between cohorts by repeating the analysis in 3 scenarios: 1) Including patients who had died by the time of the analysis: The rationale for this analysis is to assess whether differences in death rates between patients with COVID-19 and patients with influenza or other respiratory tract infections might explain the contrast between these cohorts. For instance, it might be that the more severely affected patients from one cohort (who are more likely to develop neurological or psychiatric sequelae) are more likely to survive than the more severely affected patients from another cohort. This analysis was simply achieved by removing the exclusion criterion 'Deceased on or before December 13, 2020' from the definition of cohorts.
2) Restricting the COVID-19 diagnoses to those who had a positive RNA/Antigen test (and using the latter as an index event): The rationale for this analysis is that a positive RNA/Antigen test better anchors the index event at the time when the virus was detected. The reason for not including it in the primary analysis is that diagnostic codes (rather than test results) are used for the definition of control cohorts and it therefore appears more valid to use a diagnostic code for the COVID-19 cohort as well.
In addition, patients who had both a diagnostic code and a test result recorded might be a seen in HCOs which are more meticulous in their coding of diagnoses/tests and this might inflate the rate of neurological/psychiatric sequelae thus artefactually increasing the contrast between cohorts.
3) Comparing the rates of sequelae with those observed in patients with influenza before the pandemic: In the primary analysis, we compared the rate of psychiatric sequelae between COVID-19 and influenza within the period from January 20 to December 13, 2020. This aimed to compare health events which all occur within the same time window. However, it might be that in 2020 influenza was particularly severe (e.g. if only patients with the most severe clinical presentation sought medical attention and received the diagnosis) or particularly mild (e.g. if patients at risk of having severe influenza were benefiting from lockdown and social distancing implemented in response to the COVID-19 pandemic). As a result, the influenza cohort might not be representative of patients with influenza in a pre-pandemic context. To test whether this had an effect on the contrast between groups, in this sensitivity analysis we compared the rate of neurological/psychiatric sequelae of COVID-19 (diagnosed between January 20 and December 13, 2020) to those of influenza diagnosed between January 20 and December 13, 2019 (rather than 2020).

Details on statistical analyses
In propensity score matching, the propensity score was calculated using a logistic regression (implemented by the function LogisticRegression of the scikit-learn package in Python 3.7) including each of the covariates mentioned above. To eliminate the influence of ordering of records, the order of the records in the covariate matrix were randomised before matching.

Testing proportional hazards
The assumption that the hazards were proportional when accounting for the two phases was tested using the generalized Schoenfeld approach implemented in the cox.zph function of the survival package (version 3.2.3) in R. If the proportional hazard assumption was found to be violated, then the time-varying HR was assessed using natural cubic splines (in log-time) to the log-cumulative hazard 8 . This was achieved using the generalized survival models of the rstpm2 package (version 1.5.1) in R 9 . As recommended by Royston and Parmar 8 , splines with 1, 2, and 3 degrees of freedom were estimated for both the baseline log-cumulative hazard and its cohort dependency and the number of degrees of freedom leading to the lowest Akaike Information Criterion (AIC) was selected. This was achieved on a per-comparison basis so that more complex time dependency (i.e. higher number of degrees of freedom) could be selected for a specific comparison if there was enough evidence in the data to support such complexity.
Protocol. Because of the time sensitivity of these data, a protocol was not produced prior to the analyses. Figure S1 -Baseline characteristics of the subgroups of patients with COVID-19 who developed any vs. none of the neurological/psychiatric outcomes in the 6-months after their diagnosis. These estimates were based on patients who had their diagnosis of COVID-19 on or before August 19, 2020 thus allowing at least 6 months of follow up until February 19, 2021 when this analysis was performed.    Figure S1. Shaded areas represent 95% confidence intervals.  Figure S3 -Kaplan-Meier curves for the comparison between the cohort of patients with COVID-19 and a matched cohort of patients with influenza for the same outcomes as in Figure 1 of the main manuscript. Shaded areas represent 95% confidence intervals.

COVID−19 Influenza
Mood/Anxiety/Psychotic disorder (any) Figure S4 -Kaplan-Meier curves for the comparison between the cohort of patients with COVID-19 and a matched cohort of patients with influenza for the outcomes not included in Figure S2. Shaded areas represent 95% confidence intervals.

COVID−19 Urolithiasis
Mood/Anxiety/Psychotic disorder (any) Figure S7 -Kaplan-Meier curves for the comparison between the cohort of patients with COVID-19 and a matched cohort of patients with fracture of a large bone for the same outcomes as in Figure 1 of the main manuscript. Shaded areas represent 95% confidence intervals.      Figure S15 -Kaplan-Meier curves for the comparison between the cohort of patients with COVID-19 who did not require hospitalization (red) and a matched cohort of patients with influenza who did not require hospitalization (blue) for the same outcomes as in Figure 1 of the main manuscript. Shaded areas represent 95% confidence intervals. Figure S16 -Kaplan-Meier curves for the comparison between the cohort of patients with COVID-19 who did not require hospitalization (red) and a matched cohort of patients with other respiratory tract infections who did not require hospitalization (blue) for the same outcomes as in Figure 1 of the main manuscript. Shaded areas represent 95% confidence intervals. Figure S19 -8-weekly incidence of index events (left) and two representative outcomes (right) over the study timeframe (expanded to December 20, 2020 to keep 8-weekly intervals throughout). Unlike the rates of control events, the rates of COVID-19 diagnoses mostly increased over the study timeframe. As the control events were diagnosed at a time when more outcome diagnoses were made (a well-established by-product of the pandemic), this might negatively affect the contrast between the cohorts and might imply that the reported hazard ratios are underestimated. Psychiatric illness Ischaemic stroke

Supplementary tables
Table S1 -All baseline characteristics used in the propensity score matching. In addition, the incidence of neurological or psychiatric sequelae except dementia is presented. This is informative because patients who had a diagnosis of dementia (e.g. Unspecified dementia) before their diagnosis of COVID-19 and then go on to have another dementia diagnosis (e.g. Vascular dementia) would count towards the incidence of any diagnosis (alongside those for whom this would be their first diagnosis of dementia). The incidence of 'any first' diagnosis (presented in Table 1 of the main manuscript) is not affected by this limitation since it excludes all previous diagnoses of dementia.  Table S2 -Incidence of neurological and psychiatric outcomes within 6 months after a diagnosis of COVID-19 separately estimated by gender, race and age group    Table S5 -Hazard ratios and log-rank p-values estimated for the comparison between the COVID-19 cohort and the other two matched cohorts for subcategories of the main outcomes presented in Table 2 of the main manuscript. The corresponding ICD-10 codes are displayed in brackets.               (a) Report the numbers of individuals at each stage of the study (e.g., numbers potentially eligible, examined for eligibility, confirmed eligible, included in the study, completing follow-up, and analysed) (b) Give reasons for non-participation at each stage. (c) Consider use of a flow diagram Results, Table 1,  Supplementary  Tables S1, S3, S4, S7, S9-S13 RECORD 13.1: Describe in detail the selection of the persons included in the study (i.e., study population selection) including filtering based on data quality, data availability and linkage. The selection of included persons can be described in the text and/or by means of the study flow diagram.
Methods, Results, Table 1,  Supplementary  Tables S1, S3, S4, S7, S9-S13 Descriptive data 14 (a) Give characteristics of study participants (e.g., demographic, clinical, social) and information on exposures and potential confounders (b) Indicate the number of participants with missing data for each variable of interest (c) Cohort studysummarise follow-up time (e.g., average and total amount)