{'fraction_non_alphanumeric': 0.04192609363629013, 'fraction_numerical': 0.014271852510713997, 'mean_word_length': 5.232048374905518, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 3, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Is There Evidence for the Specificity of Closed-Loop Brain Training in the Treatment of Internalizing Disorders? A Systematic Review

Introduction Internalizing disorders (IDs), e.g., major depressive disorder (MDD), posttraumatic stress disorder (PTSD), obsessive-compulsive disorder (OCD) are the most prevalent psychopathologies experienced worldwide. Current first-line therapies (i.e., pharmacotherapy and/or psychotherapy) offer high failure rates, limited accessibility, and substantial side-effects. Electroencephalography (EEG) guided closed-loop brain training, also known as EEG-neurofeedback (EEG-NFB), is believed to be a safe and effective alternative, however, there is much debate in the field regarding the existence of specificity [i.e., clinical effects specific to the modulation of the targeted EEG variable(s)]. This review was undertaken to determine if there is evidence for EEG-NFB specificity in the treatment of IDs. Methods We considered only randomized, double-blind, sham-controlled trials. Outcomes of interest included self/parent/teacher reports and clinician ratings of ID-related symptomatology. Results Of the four reports (total participant number = 152) meeting our eligibility criteria, three had point estimates suggesting small to moderate effect sizes favoring genuine therapy over sham, however, due to small sample sizes, all 95% confidence intervals (CIs) were wide and spanned the null. The fourth trial had yet to post results as of the submission date of this review. The limited overall number of eligible reports (and participants), large degree of inter-trial heterogeneity, and restricted span of ID populations with published/posted outcome data (i.e., PTSD and OCD) precluded a quantitative synthesis. Discussion The current literature suggests that EEG-NFB may induce specific effects in the treatment of some forms of IDs, however, the evidence is very limited. Ultimately, more randomized, double-blind, sham-controlled trials encompassing a wider array of ID populations are needed to determine the existence and, if present, degree of EEG-NFB specificity in the treatment of IDs. Systematic Review Registration [https://www.crd.york.ac.uk/prospero], identifier [CRD42020159702].

That said, skeptics assert that EEG-NFB's efficacy derives exclusively from non-specific factors (e.g., expectations, demand characteristics, and context) based primarily on a collection of randomized, sham/placebo-controlled trials for attention deficit hyperactivity disorder (ADHD) which demonstrated comparable clinical improvements in both experimental and sham groups (Thibault et al., 2016(Thibault et al., , 2018Schönenberg et al., 2017a,b;Ghaziri and Thibault, 2019;Arnold et al., 2021). EEG-NFB proponents' most salient objection to this conclusion is that evidence of differential targeted EEG-learning (i.e., greater improvement in the trained electrophysiological variable(s) in genuine vs. sham EEG-NFB groups), considered by many to be essential for a valid evaluation of EEG-NFB's specificity (Sherlin et al., 2011;Kerson, 2013;Arns et al., 2014;Holtmann et al., 2014;Zuberer et al., 2015;Szewczyk et al., 2018;Witte et al., 2018), was noticeably absent in the trials presented as evidence for wholly non-specific effects (Pigott et al., 2018;Trullinger et al., 2019). Remediation of this apparent shortcoming can be complicated, however, due to a lack of established guidelines for determining successful EEG-learning (Weber et al., 2020).
The aim of our review was to comprehensively evaluate all available randomized, double-blind, sham/placebo-controlled trials in an ID population for evidence of EEG-NFB specificity via differences in clinical outcomes (i.e., symptom rating scales) between genuine and sham EEG-NFB groups.

Registration and Protocol
This review was prospectively registered on the International Prospective Register of Systematic Reviews (PROSPERO) under the registration number: CRD42020159702. The protocol for this systematic review and meta-analysis has been published previously (Perez et al., 2021) and uploaded in PDF format to PROSPERO.

Eligibility Criteria
We considered all EEG-NFB reports involving humans with at least one ID diagnosis per the Diagnostic and Regarding outcome assessments, all reports were required to include data from at least one clinical rating scale assessing one or more core symptoms of the disorder(s) under investigation. To minimize bias and control for non-specific effects, we included only randomized, double-blind (participants and outcome assessors), sham/placebo-controlled (i.e., feedback contingent on either a random signal, the activity from a different person's brain, or a signal from the participant's own brain derived from a region unrelated to the condition under study) reports. Concomitant interventions were permitted provided they were identical for both active and sham groups.

Information Sources
Studies eligible for review were identified in a literature search from earliest dates within multiple databases ( Table 1). Of note, we decided to remove database limit options from our protocol in order to broaden the scope of our searches. Databases were last searched on 23 November 2021. Considering the known importance of including unpublished data in systematic reviews (Trespidi et al., 2011), we also searched various clinical trial registries including ClinicalTrials.gov 1 , the World Health Organization's International Clinical Trials Registry Platform (ICTRP) 2 , and the Australia New Zealand Clinical Trials Registry (ANZCTR) 3 to identify completed, unpublished trials. Notably, the ICTRP indexes trial registrations from 17 registries around the world. All registries were last accessed on 23 November 2021. Additionally, reference lists of included articles and relevant systematic reviews were manually screened to identify additional studies.

Search Strategy
Literature search strategies were developed using medical subject heading (MeSH) and text words related to IDs and neurofeedback. The search strategies were developed by author TMP with guidance from the University of Otago's Health Sciences librarian. A detailed account of the search strategy for each database and registry can be found in Supplementary Material.

Selection Process
TMP collated the list of possible records for inclusion and exported them from each database to EndNote (version X9.2) where duplicates were located using EndNote's duplicate identification strategy (i.e., identifying references in a library of the same reference type with matching Author, Year, and Title fields) and then removed manually. Two independent reviewers (TMP and JM) screened titles and abstracts for eligibility. In cases of disagreement, consensus on articles to assess for eligibility was reached by discussion between TMP and JM. When disagreements couldn't be resolved, a third team member (DBA) was enlisted to make the final decision. TMP and JM then independently assessed full-text reports appearing to meet the inclusion criteria or when there was any uncertainty. Again, in cases of disagreement, consensus on inclusion/exclusion was reached via discussion and, when needed, DBA was consulted to make the final decision. When necessary to resolve questions regarding eligibility, TMP sought additional information from study authors via a maximum of three electronic (i.e., email and/or ResearchGate) requests.
Notably, two non-English language reports [ (Biriukova et al., 2005;Eskandari et al., 2014)] were translated during our full-text eligibility assessments. One report [(Biriukova et al., 2005)] was entered into Google Translate by our team and the subsequent output was validated by the study's last author. The other (Eskandari et al., 2014) was translated by a native speaker here at the University of Otago. Reasons for excluding trials were recorded. Neither TMP nor JM were blinded to the journal titles, trial authors, or institutions.

Data Collection Process
Data was extracted by TMP and independently verified by JM.
In cases of disagreement, consensus was reached via discussion. For all three included reports, trial authors were successfully contacted via email for clarification of trial details and/or to obtain missing data.

Data Items
Our primary outcome of interest was clinician ratings or self/parent/teacher reports of ID-related symptomatology. In trials incorporating multiple domains (e.g., clinician ratings and self/parent/teacher reports), a single scale was selected based on a hierarchy (i.e., clinician > self > parent > teacher). When multiple rating scales within a given domain were assessed, the validated scale querying the most core/central feature(s) of the condition under study (as determined by our content expert PG) was selected. In cases where multiple values for a single scale (i.e., total vs. sub-scale scores) were assessed, the total scores were used. Notably, as it has been postulated that longer-term outcomes may help to clarify the issue of specificity (Van Doren et al., 2019), in trials with multiple post-treatment data collection time-points, scores obtained furthest from treatment termination were given preference. To date, standard EEG-NFB protocols have not been established for the treatment of IDs (Banerjee and Argáez, 2017), therefore, no protocols were excluded.

Study Risk of Bias Assessment
The risk of bias for each eligible report was assessed using the Cochrane Risk of Bias tool version 2 (RoB 2.0) for randomized trials which covers 5 domains (domain 1: risk of bias arising from the randomization process; domain 2: risk of bias due to deviations from the intended interventions; domain 3: risk of bias due to missing outcome data; domain 4: risk of bias in measurement of the outcome; domain 5: risk of bias in the selection of the reported result) (Sterne et al., 2019). Two reviewers (TMP and JM) independently applied the tool to each trial and recorded supporting information/justifications for risk of bias judgments (low, some concerns, high) in each domain. Further, an overall summary risk of bias judgment (low, some concerns, high) was made for each report determined by the highest risk of bias level across all 5 domains. All decisions were guided by the published criteria for judging the risk of bias . If there was insufficient detail reported in the study, the original study investigators were contacted via email for more information. Disagreements were resolved in discussion and, in cases where consensus was not reached, a third team member (DBA) was enlisted to make the final decision.

Effect Measures
Because the included trials utilized different measurement scales to assess clinical outcomes, standardized mean differences (SMDs) and 95% confidence intervals (CIs) were calculated using Hedges' adjusted g in RevMan (version 5.4.1) and presented in a forest plot. In our forest plot, an SMD of 0 is represented by a black vertical line with negative (left-sided) and positive (rightsided) values favoring sham and active groups, respectively. Each trial's SMD point estimate and 95% CI are represented by a green vertical dashes and bilateral black horizontal lines, respectively. The SMD expresses the size of the intervention effect relative to the variability observed under the assumption that between-study standard deviation variations reflect differences in measurement scales rather than variability in the study populations or reliability in outcome measures (Deeks et al., 2021). SMDs of 0.2, 0.5, and 0.8 are generally interpreted to reflect small, medium, and large effects sizes, respectively (Cohen, 1988).

Synthesis Methods
The limited overall number of eligible reports (and participants), large degree of inter-trial heterogeneity, and restricted span of ID populations with published/posted outcome data (i.e., PTSD and OCD) precluded a robust quantitative synthesis investigating the potential for EEG-NFB specificity in the treatment of IDs.

Reporting Bias Assessment
To assess for reporting biases (e.g., selective non-publication and selective non-reporting of results), two independent reviewers (TMP and JM) searched various registries for unpublished trials as well as to compare published trial report outcomes to outcomes specified in their registrations to help guide ratings for domain 5 (i.e., risk of bias in the selection of the reported result) of the RoB 2.0. When registrations were unavailable, we compared the congruency of the section "Materials and Methods" and section "Results" for each trial. Disagreements were resolved in discussion and, when necessary, DBA was consulted to make the final decision.

Certainty Assessment
Two independent reviewers (TMP and JM) assessed the certainty in the body of the evidence as it related to the trials that contributed data to the meta-analysis using the Grading of Recommendations, Assessment, Development and Evaluations (GRADE). A certainty ranking of high (there is a lot of confidence that the true effect lies close to that of the estimated effect), moderate (the true effect is probably close to the estimated effect), low (the true effect might be markedly different from the estimated effect.), or very low (the true effect is likely to be substantially different from the estimated effect) was assigned by the software based on the reviewers assignments across a number of domains including study design (i.e., randomized trial or observational study), risk of bias (not serious, serious, or very serious), inconsistency (not serious, serious, or very serious), indirectness (not serious, serious, or very serious), imprecision (not serious, serious, or very serious). We used the methods and recommendation described in the GRADE handbook (Schünemann and Oxman, 2013). Disagreements were resolved in discussion and, when necessary, DBA was consulted to make the final decision. A GRADE profile (v2) was generated using GRADEpro GDT software (Evidence Prime, 2020). Where necessary, we provided explanations for our assignments using footnotes to aid the reader's understanding of the results.

Study Characteristics
An detailed overview of the eligible trials (ClinicalTrials.gov, 2000a;Nederlands Trial Register, 2004;Kopřivová et al., 2013;Nicholson et al., 2020) is presented in Table 2. As can be seen, there is significant clinical (e.g., population) and methodological (e.g., clinical outcome scale) heterogeneity across trials.

Risk of Bias in Studies
We used the RoB 2.0 to assess risk of bias for the eligible trials with posted/published outcome data (Figure 2). The overall risk of bias was deemed moderate (i.e., "some concerns") for two of the trials (Kopřivová et al., 2013;Nicholson et al., 2020) and high for the third (ClinicalTrials.gov, 2000a) suggesting some methodological heterogeneity across trials. Concerns included a lack of pre-registered protocols and/or analysis plans with which to compare the final reports and outcomes.

Results of Individual Studies
Figure 3 displays summary statistics [CFB mean and SD, sample size (Total)] for the sham and genuine EEG-NFB groups along with the SMD and 95% CI of the continuous outcome (reduction in internalizing symptoms) for each study. Of note, one eligible trial [i.e., (Nederlands Trial Register, 2004)] is absent due to a failure to post/publish results. Unfortunately, the lead trialist was unwilling to share unpublished data. For all trials with final results, trialists were successfully contacted via email for additional data not provided in the original report. Baseline outcome measures were reported to be equivalent between genuine and sham groups in the two published studies (Kopřivová et al., 2013;Nicholson et al., 2020) and were not reported in the unpublished trial (ClinicalTrials.gov, 2000a (Nicholson et al., 2020). The two published trials [i.e., (Nicholson et al., 2020)    and (Kopřivová et al., 2013)] did assess for differential targeted EEG-learning with group point estimates suggesting a trend toward greater improvements (i.e., reductions) in the trained EEG variable(s) in the genuine EEG-NFB groups relative to sham, however, correlation analyses between changes in the targeted EEG variable(s) and clinical outcome measures were not performed.

Reporting Biases
Overall, we feel the risk of selective outcome reporting was low as all trials' methods and results sections were congruent and wellaccepted scales were generally incorporated for the conditions under study, however, there remains a risk of selective analysis reporting due to a lack of pre-specified analysis plans per trial registry records or published protocols. Additionally, we have concerns surrounding publication bias as we were unable to include the outcome data from one of the completed trials meeting our eligibility criteria [i.e., (Nederlands Trial Register, 2004)] because they have yet to be made publicly available despite being "terminated [in 2017] because of disappointing mid trial results" (personal communication with the lead trialist, 7 July 2020).

Certainty of Evidence
As seen in the GRADE profile (Table 3), although the certainty of the evidence is initially rated "high" due to the incorporation of the three RCTs with published/posted outcomes, it was subsequently downgraded three steps to "very low" due to (1) "serious" concerns surrounding the cumulative risk of bias, (2) "serious" concerns regarding the imprecision (i.e., wide 95% CIs), and (3) "strongly suspected" publication bias (i.e., no published/posted outcomes from the fourth eligible trial).

DISCUSSION
To our knowledge, this is the first systematic review exploring the potential for EEG-NFB specificity (i.e., specific effects) in FIGURE 3 | Forest plot showing standardized mean differences in change-from-baseline scores between sham and genuine EEG-neurofeedback using a Hedges' (adjusted) g. ID populations. Of the four eligible reports identified, three (2 PTSD and 1 OCD) trials provided outcome data with point estimates suggesting EEG-NFB specificity, however, effect sizes were modest with wide 95% CIs spanning the null. The fourth trial, which recruited people with MDD, has yet to publish its results although, according to the lead trialist, the results were "disappointing." Notably, it could be argued that the protocols undertaken in the eligible trials may have mitigated betweengroup differences and the overall effect size. More specifically, it has been reported that individualized training is typical in clinical settings and may lead to augmented effects and better outcomes (Hammond, 2010;Omejc et al., 2019;Weber et al., 2020), however, only one of the trials utilized an individualized training paradigm insofar as the targeted frequency or frequency band was based on individual deviations from a normative sample. Furthermore, it has been postulated that non-specific (i.e., placebo) effects are transient, therefore longer term followups serve to better elucidate specificity (Van Doren et al., 2019) yet only one of the eligible trials incorporated any follow-up (i.e., 3 months). Although highly speculative, against this general backdrop of standardized protocols and lack of extended follow ups, theoretically the effect sizes may have been downwardly biased. Some limitations of the current review should be noted. Firstly, although we had no exclusions by language, we may have unintentionally overlooked some non-English language trials meeting our criteria considering the fact that coverage bias (i.e., systematic exclusion of journals from certain countries and/or in certain languages) has been reported in some of the databases that were utilized (Pilkington et al., 2005). That said, the exclusion of non-English language publications from quantitative reviews of clinical interventions has been shown to have little-to-no effect on results and conclusions (Nussbaumer-Streit et al., 2020). Secondly, although two of the trials did assess for differential group EEG-learning (i.e., reduction in the power of the targeted frequency band) with point estimates in both suggesting greater improvements in the genuine group relative to sham, neither correlated this learning with clinical outcomes. Thirdly, outcome data were only available for PTSD and OCD populations, thereby excluding information on the most prevalent IDs (i.e., anxiety and depressive disorders) and limiting the generalizability of our findings. Lastly, due to the small number and extreme heterogeneity of eligible trials, a proper meta-analysis could not be undertaken. A major potential contributor to these latter two limitations was that many candidate trials failed to provide sufficient details necessitating requests for additional information and, in many cases, exclusion of their data due to non-responsive trialists. Reporting issues in RCTs are widespread (Moher et al., 2012) which have prompted skeptics and proponents alike to come together to publish a consensus paper on the reporting and design of neurofeedback studies . It is our hope that these guidelines will be widely adopted in the field and incorporated into future publications. Additionally, the lack of eligible reports, small participant numbers, and lack of replication may be explained, in part, by the significant investment in resources (i.e., time, human, and monetary) required to undertake EEG-NFB trials of this nature although this too is highly speculative.
Neuropsychiatric disorders are among the most common causes of morbidity and mortality (Kessler et al., 2009) with rates markedly increasing world-wide in recent years Keyes et al., 2019;Twenge et al., 2019;Haidt and Allen, 2020;Pfeifer and Allen, 2021). Among them, IDs, which are characterized by distress experienced inwardly (Cosgrove et al., 2011;Buchan et al., 2014), are the most prevalent. Recently, a government inquiry here in Aotearoa New Zealand has shed light on the shortcomings of traditional front-line treatments (i.e., pharmacotherapy and/or psychotherapy) and urged wider implementation of alternative approaches to the treatment of mental health problems (Kris, 2018). Likewise, scientists abroad have called for more research into "novel interventions that may be based on altering plasticity or returning circuitry rather than neurotransmitter pharmacology" (Insel and Wang, 2010). EEG-NFB is a non-invasive treatment that appears to be generally safe and efficacious when directed by a qualified practitioner (Hammond and Kirk, 2008), however, despite over 50 years since its inception, questions regarding the nature of those effects abound. Specifically, there is much controversy surrounding the existence of EEG-NFB specificity [i.e., specific effects deriving from modulation of the EEG variable(s) of interest]. The current systematic review suggests that genuine EEG-NFB may induce specific effects in at least some ID populations (i.e., PTSD and OCD). That said, the evidence is very limited, thereby warranting more randomized, double-blind, sham-controlled studies to verify the existence and, if present, degree of specificity across the spectrum of IDs. Encouragingly, we are aware of a number of registered randomized, double-blind, sham-controlled EEG-NFB trials in ID populations [i.e., (ClinicalTrials.gov, 2000b;Australian New Zealand Clinical Trials Registry, 2005a,b)] which, when incorporated into a future meta-analysis, may help bring clarity to this intriguing and important topic.

DATA AVAILABILITY STATEMENT
The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding author.

AUTHOR CONTRIBUTIONS
TP is the guarantor and drafter of the manuscript and developed and implemented the search strategies. PG provided expertise on mental health disorders. JM assisted with the screening, data extraction, and assessments. All authors contributed to the development of the selection criteria, the risk of bias assessment strategy and data extraction criteria with guidance from the Health Sciences Librarian and read, provided feedback, and approved the final manuscript.

FUNDING
This systematic review is part of a Ph.D. thesis supported by the Department of Surgical Sciences, University of Otago, Dunedin, New Zealand.

ACKNOWLEDGMENTS
First, we would like to acknowledge the University of Otago Health Sciences Librarian, Christy Ballard, for her help with designing the search strategies. Second, we would like to thank Mark Trullinger, who graciously agreed to proofread and provide feedback on the manuscript. Lastly, we are grateful to all the trialists who responded to our requests for additional information.