{'fraction_non_alphanumeric': 0.025165246476069124, 'fraction_numerical': 0.006185129144434711, 'mean_word_length': 5.367816091954023, 'pattern_counts': {'":': 1, '<': 0, '<?xml version=': 0, '>': 2, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
The relative effectiveness of psychotherapeutic techniques and delivery modalities for chronic pain: a protocol for a systematic review and network meta-analysis.

Introduction: There is increasing evidence for the use of psychotherapies, including cognitive behavioural therapy, acceptance and commitment therapy, and mindfulness based stress reduction therapy, as an approach to management of chronic pain. Similarly, online psychotherapeutic interventions have been shown to be efficacious, and to arguably overcome practical barriers associated with traditional face-to-face treatment for chronic pain. This is a protocol for a systematic review and network meta-analysis aiming to evaluate and rank psychotherapies (delivered in person and online) for chronic pain patients. Methods/ design: Four databases, namely the Cochrane Central Register of Controlled Trials (CENTRAL), MEDLINE, EMBASE and PsycINFO will be searched from inception. Randomised controlled trials that have evaluated psychological interventions for pain management delivered online or in person will be included in the review. Data will be independently extracted in duplicate and the Cochrane Collaboration Risk of Bias Tool will be used to assess study quality. Measures of pain interference will be extracted as the primary outcome and measures of psychological distress will be extracted as the secondary outcome. A network meta-analysis will generate indirect comparisons of psychotherapies across treatment trials. Rankings of psychotherapies for chronic pain will be made available. Discussion: A variety of psychotherapies, delivered both online and in person, have been used in an attempt to help manage chronic pain. Although occasional head to head trials have been conducted, little evidence exists to help identify which psychotherapy is most effective in reducing pain interference. The current review will address this gap in the literature and compare the psychotherapies used for internet delivered and in person interventions for chronic pain in relation to the reduction of pain interference and psychological distress. Results will provide a guide for clinicians when determining treatment course and will inform future research into psychotherapies for chronic pain. PROSPERO registration: CRD42016048518 01/11/16


Introduction
Chronic pain Chronic pain refers to pain ("an unpleasant sensory and emotional experience associated with actual or potential tissue damage, or described in terms of such damage" [IASP, 2012;Merskey et al., 1979]) that persists for more than three months. Chronic pain is highly prevalent, and one of the leading causes of long-term disability and reduced quality of life globally (Blyth et al., 2019;Hurwitz et al., 2018;Raftery et al., 2011). Chronic pain is considered a highly multifaceted disorder with multiple physical, emotional, behavioural and cognitive determinants (Lumley et al., 2011). Although pharmacological treatments can reduce pain severity, they do not yield concurrent improvements in reducing disability and emotional distress; psychological therapies are typically implemented in conjunction with medical treatments to yield the highest improvements in physical and emotional wellbeing (Turk et al., 2008).

Psychotherapies for chronic pain
Psychological therapies for chronic pain typically focus on addressing the cognitive, behavioural, emotional and social factors thought to sustain physical disability (Sturgeon, 2014). Cognitive behavioural therapy (CBT) is generally accepted as a treatment for chronic pain and is currently considered the "gold standard" in psychological interventions for pain (Day et al., 2012). Research has contributed to the emergence of new psychotherapeutic approaches for chronic pain management including Acceptance and Commitment Therapy (ACT; Wetherell Why is it important to do this review? Psychotherapies are generally accepted as a treatment option for chronic pain patients. CBT has historically been considered the gold standard treatment, however, with research into different psychotherapies yielding positive results, it is difficult to know the most appropriate direction in either a clinical or research setting. There has also been an increase in internet delivered interventions for chronic pain, however, there is a dearth of research evaluating the efficacy of online psychotherapies in comparison to their counterparts delivered in person. The current review will investigate the current available evidence using a network-meta analysis (NMA). In the context of a systematic review, NMA is a statistical technique enabling comparisons of multiple treatments through direct and indirect comparisons across trials that use a common comparator (for further discussion, see Jansen et al., 2014;Naci et al., 2013). This will facilitate comparisons regarding the relative efficacy of different psychotherapies and delivery modalities that would otherwise not be possible until further primary research is carried out.

Amendments from Version 1
Version 2 of this article incorporates changes based on feedback from reviewers. We have edited the title of the paper to better reflect its aims, reformulated our objective to be clearer about our research question, and added further detail to the analysis section to clarify the exact process we will follow (in particular around assessing the risk of bias and quality of evidence).
Any further responses from the reviewers can be found at the end of the article

Objective
The objective of this study is to evaluate the relative effectiveness of existing online and in-person psychotherapeutic interventions for chronic pain management. This will be done by reviewing the current available evidence from relevant papers and using the NMA method to create a network of direct and indirect comparisons, thereby creating a tentative ranking of interventions.

Protocol and registration
This systematic review and NMA will be conducted and reported in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines and the PRISMA Network Meta-Analysis extension statement (see Reporting guidelines; Hutton et al., 2015;Moher et al., 2009). In accordance with these recommendations, this review will use PICO to frame and report eligibility criteria; as such, participants, interventions, comparisons, outcome(s) and study design of included studies will be reported. The protocol for this study was registered with the International Prospective Register of Systematic Reviews (PROSPERO) database (registration number: CRD42016048518) on 1 November 2016.

Types of studies
Only randomised controlled trials (RCTs) comparing a psychological intervention, with at least one of an alternative psychological intervention, waitlist control (WLC), treatment-as-usual (TAU) or non-psychological intervention (for example, exercise, education and medical therapy) will be eligible for inclusion. Eligible psychological interventions must have identifiable psychotherapeutic content such as behavioural (for example, biofeedback, relaxation or behaviour monitoring) or cognitive behavioural (for example, coping skills, cognitive re-constructing or problem solving) components. The interventions must be delivered in person or via the internet. There are no restrictions placed on treatment intensity, duration or number of psychotherapeutic techniques employed. Studies must be full-text journal articles available in English, published in peer-reviewed journals, and available using database access or through contacting study authors.

Types of participants
Eligible participants must be adults (>18 years) living with chronic pain, as defined by the International Association for the Study of Pain (IASP 2012;Merskey et al., 1979). Studies which examine samples where chronic pain is induced by debilitating diseases such as cancer or multiple sclerosis will not be included due to differences in disease prognosis compared to other forms of chronic pain (Treede et al., 2015). As chronic headaches and migraine treatment is generally considered to be different to other chronic pain treatments (Williams et al., 2012), studies examining samples where headache is the primary disorder will not be included.
Types of outcome measures Primary outcomes. Studies will only be included in the NMA if they contain a self-reported measure of pain related interference or similar, for example pain related disability or impact.

Secondary outcomes.
A second network will be created to investigate the secondary outcome which will include selfreported scales of psychological distress (including depression, anxiety, negative affect or psychological stress).

Electronic searches
The following databases will be searched from inception: EMBASE, MEDLINE, CENTRAL (Cochrane Library), and PsycINFO. Searches strategies will be the same for all databases; however necessary changes will be made to account for differences in interfaces. The search strategy is described in Table 1 (see extended data (O'Connor, 2019)).

Searching other resources
Reference lists of relevant systematic reviews will be searched to identify additional relevant studies. The metaRegister of Controlled Trials (mRCT), clinicaltrials.gov and the WHO International Clinical Trials Registry Platform (ICTRP) will also be searched. This review will only include studies published in peer-reviewed journals; dissertations, unpublished papers and on-going studies will be excluded.

Study selection
Search results will be imported into Endnote X7 to create a library of all studies found, and individual folders created to track included and excluded studies. The research team will screen titles and abstracts, first to identify duplicate studies and then screen for any studies that do not satisfy the inclusion criteria, with 10% of studies screened in duplicate to ensure consistency. Studies not available in English will be excluded at this time. Two authors (SQ and CJ) will independently screen full papers of the remaining studies for inclusion in agreement with the inclusion criteria. Studies will be sequentially excluded via the exclusion categories if they do not satisfy the criteria. Disagreements between screeners will be discussed, and where a decision cannot be reached, a third reviewer (SH) will mediate. A flow chart will graphically depict the exclusion of studies.
Data collection process Data will be extracted from the included studies in duplicate into a pre-prepared data extraction Excel sheet, to be piloted with three studies and amended if necessary before data extraction begins proper. Authors of included papers will be contacted if it is necessary to recover missing data.
Data items Data will be extracted in accordance with the following categories: • Participants: sample size, percentage female, mean age, diagnosis, mean years of pain.
• The type of psychotherapy/control employed in each arm.
• Primary measure used to record each outcome.
• Means and standard deviations at post intervention of suitable outcome measures and the measure used to collect the data.
Geometry of the network A network diagram will visually represent available evidence, depicting the possible comparisons of any two psychotherapies (for example, CBT vs ACT) and the comparison of delivery methods of psychotherapy (for example, CBT vs iCBT). Any arms not directly connected to the network will not be included in analysis.

Risk of bias in included Studies
As participants and intervention providers are unable to be blinded in studies assessing psychological interventions, the bias domain assessing blinding of participants/personnel will be omitted from this review. Similarly, as this review focuses solely on self-report measures, the bias domain assessing blinding of outcome assessors will also be omitted. Risk of bias will be assessed by two independent reviewers. Two authors (JF & BO'G)will independently use the Cochrane Collaboration Risk of Bias tool to classify studies as being of low, high or an unclear risk of bias based on the following six domains; random sequence generation, allocation concealment, incomplete outcome data, selective reporting bias, use of intention-to-treat analysis, uneven distribution of potential cofounders at baseline. Where disagreements regarding the risk of bias are encountered, a third author (SH) will mediate. For more detail see Slattery et al. (2017).

Quality of evidence
The Grading of Recommendations Assessment, Development and Evaluation (GRADE) criteria will be used to judge the quality of evidence for the direct, indirect and NMA effect estimates yielded within the network (Puhan et al., 2014). The quality of the direct effect estimates will be judged based on the methodological limitations of the included trials, imprecision, indirectness, inconsistency, and publication bias. Providing the certainty of the direct estimates is high, and the contribution of the direct estimates is at least equal to that of the indirect estimates, the quality of evidence for the network estimates will be based solely on the direct evidence (Brignardello-Petersen et al., 2018). Otherwise, the quality of the indirect evidence will be determined using the direct comparisons with the highest contribution in generating those comparisons, and the quality of the network estimates will be determined based on the quality of the direct and indirect comparisons making the largest contribution. In this instance, imprecision will not be considered when evaluating the quality of the network estimates (Brignardello-Petersen et al., 2018). Additionally, where incoherence between direct and indirect estimates is evident, the quality of the estimate making the highest contribution will be used to inform the quality of the network estimate but will be downgraded regardless (Brignardello-Petersen et al., 2018). In the instance of sparse networks and incoherence resulting in widened network estimate confidence intervals, exploratory sensitivity analyses based on the removal of trials with common covariance will be considered (Brignardello-Petersen et al., 2019).

Summary measures
This review will use Stata 13 for all analyses. Mean differences between groups and 95% confidence intervals will be reported. Where outcomes are obtained using a variety of measures, standardised mean differences (SMD) will be calculated and reported with their 95% confidence intervals. To adjust for the SMD direction, the mean score of measures in which higher scores are indicative of lower levels of the desired construct (for example, higher scores on the SF-36 indicate lower levels of disability) will be subtracted from the maximum value of that measure (Higgins & Green, 2008). If no standard deviations are reported, they will be calculated from standard errors or confidence intervals that are available. Where suitable data are unavailable, study authors will be contacted in an attempt to retrieve data. If these data are not retrievable, such studies will not be included.

Planned methods of analysis
This review will generate comparisons of psychotherapies delivered online and in person with a view to determining which psychotherapy most effectively reduces pain interference and psychological distress respectively in a chronic pain population.

Exploratory analysis (Pair-wise meta-analyses).
Where data are available, pairwise meta-analyses (using Stata 13) of each comparison will act as an exploratory analysis. In the event that significant heterogeneity is discovered, a random effects model will be used. Forest plots will be used to illustrate the individual and pooled effect sizes.
Network meta-analysis. A random effects NMA will be generated on Stata 13. The model will take an exploratory frequentist approach, whereby the parameter estimates observed within each study are used to make inferences regarding the relative efficacy of each treatment without priori hypotheses. Pooled SMDs and 95% confidence intervals will respectively be used to quantify the magnitude of difference between each intervention type and the uncertainty surrounding these differences. Direct estimates will be generated based on interventions directly compared in trials, indirect estimates will be generated based on the results of direct estimates where intervention types that have not been directly compared share a common comparator, and network estimates will be generated through combining direct and indirect estimates (Al Khalifah et al., 2018). A network forest plot and interval plot will be created to graphically represent the effect size of each included study. Treatment rankings and SUCRA cumulative ranking probabilities will be created to identify the most effective psychotherapies

Classification of arms:
For the primary analysis, studies will be grouped based on the type of psychotherapy employed and the modality of delivery (for example, in person vs. online CBT). Each category must have data from more than one study to warrant inclusion in the network. Otherwise, such studies will be categorised as "other" and further subcategorised based on the delivery modality. For studies employing combinations of psychotherapeutic techniques that do not coincide with a specific intervention format, such trials will be grouped in a "multimodal" category which will be further subcategorized based on the delivery modality. As low statistical power between comparisons is typically a concern when conducting NMAs (Thorlund & Millis, 2012) and large numbers of interventions employing unconventional and unique psychotherapeutic formats are anticipated, the purpose of this categorisation procedure is to avoid low statistical power between trial comparisons. Studies utilising a medium of psychotherapy delivery that does not meet our inclusion criteria (such as over the phone or teleconferencing) will be categorised as "other delivery modalities", providing that at least one arm in that study employs an eligible delivery modality. Trial arms including WLC or TAU will be categorised as inactive controls whereas arms assessing interventions without identifiable psychological components (for example, exercise, education and medical therapy) will be categorised as active controls.

Statistical heterogeneity and inconsistency
Given the variety in study length, engagement required, and samples used, notable heterogeneity is anticipated. Statistical heterogeneity will be assessed using the I 2 statistic, which calculates the percentage of variability due to heterogeneity and not chance. Where substantial heterogeneity is evident between trials (I 2 >40%), subgroup analyses and meta-regressions will respectively be considered to determine the extent to which categorical and continuous covariates influenced the effect sizes generated, such as the type of chronic pain, length of morbidity, pain severity at baseline, and the duration of the intervention. Within the NMA, global incoherence (i.e., the extent to which the direct and indirect estimates are in agreement) will be assessed using the Wald statistic, and a local test will be conducted to investigate loop incoherence.

Risk of bias across studies
As part of the exploratory analysis, funnel plots of both outcomes will be generated in Stata 13. These plots will be assessed for symmetry to determine the presence of publication bias. The Egger Test will also be conducted using Stata 13, to investigate whether study size is related to the study estimate.

Additional analyses
The influence of studies at a high risk of bias will be investigated by removing them from the pairwise and network meta-analyses one at a time. Studies with at least one high risk of bias domain will be considered to be of high risk and removed from the analysis.

Dissemination of information
We intend on disseminating the findings through traditional academic platforms including peer-reviewed journals and academic conferences. We will also disseminate the findings through the Centre for Pain Research's public facing channels (such as our website and social media). In doing so, we hope to facilitate the dissemination of our findings to all stakeholders including clinicians, researchers, and chronic pain patients.

Study status
We have finished the search and study screening phases and are currently preparing to extract data from the included studies. Following data extraction, studies will be categorised based on the treatment type and delivery modality. Following study categorization, the analyses, quality of evidence assessment, and write up will be conducted.

Contribution to literature
It is generally accepted that psychotherapies are effective in assisting pain management for a chronic pain population. Furthermore, there has been an increase in the number and variety of psychotherapies delivered online for chronic pain. Although research has been done in this area (Tang, 2018; Vugts et al., 2018), to date, in the context of chronic pain, there is no clear indication as to which psychotherapy delivered either in person or online is the most effective. The proposed review will address this gap in the literature.
This review will extend previous research in the area by quantitatively comparing a variety of psychotherapies delivered both online and in person to identify the most efficacious in the context of chronic pain. Specifically, the NMA will return rankings, determining which psychotherapy and modality has been the most effective in reducing the primary outcome, pain interference. Such information will guide clinicians, as they choose psychotherapies to use when delivering an intervention for chronic pain.
As the treatment rankings yielded by the NMA will indicate the relative efficacy and cost-effectiveness of different psychological treatments for chronic pain management, the rankings will enable treatment providers to select the most suitable treatment options for their patients. For example, if a particular intervention is found to be very effective but expensive to implement, it would be important for clinicians to know the following most effective treatment, while keeping the costeffectiveness of each intervention into consideration. In much the same way the rankings will be an aid to researchers choosing components for chronic pain studies. Additionally, through implementing the GRADE criteria, this review will provide an indication of the methodological quality of the available research in psychotherapy for chronic pain management, and the extent to which the methodological quality could have influenced the findings.
Limitations A significant degree of heterogeneity is anticipated due to trials being grouped regardless of variations in intervention format (e.g. online, in-person, with/without therapist contact, number of sessions) or sample characteristics (e.g. age, socio-economic status, degree of pain interference, comorbidities). This could lead to high uncertainty surrounding the effect estimates and consequently bias any inferences (Riley et al., 2017). Additionally, as studies will be grouped regardless of chronic pain-type, the generalizability of the findings towards specific diagnoses will be compromised. As all studies employing multiple psychotherapeutic techniques will be categorized as "multimodal" and all studies employing unconventional psychotherapeutic strategies will be categorised as "other", this could result in further heterogeneity between the included trials. Additionally, deciphering the effective components of such categories will be impossible using this categorisation procedure. Although research examining the impact of language restrictions within meta-analyses is mixed (Jüni et al., 2002; Wang et al., 2015), this inclusion criterion introduces the possibility of language bias into the review. As only follow-up data will be synthesised, the findings will provide no insight into the long-term sustainability of the treatments. As recipients of psychological interventions are unable to be blinded to their treatment condition, all of the studies incorporated in this review are susceptible to detection and allegiance biases (Munder et al., 2011). Although measures will be taken to assess the magnitude of publication bias, only including peer-reviewed articles could increase publication bias. Additionally, as this review focuses solely on self-report measures, all included studies are susceptible to response bias or social desirability (Rosenman  et al., 2011). These inherent biases could consequently inflate the treatments' effects (Shean, 2014).

Implications of the review
This review will, to our knowledge, provide the first comprehensive overview of the relative effectiveness of various psychotherapeutic interventions for chronic pain. This review will also provide a novel understanding of how the medium of psychotherapy delivery influences the effectiveness of such interventions. These insights can inform healthcare professionals on the most effective treatment route for improving health related quality of life amongst chronic pain patients. Moreover, as online psychotherapeutic interventions have been found to be a cost-effective modality of treatment delivery (Lenhard et al., 2017) and financial constraints are typically reported barriers to accessing non-pharmacological treatment in chronic pain sufferers (Becker et al., 2017), any findings indicating preferable effectiveness of online treatment could encourage increased treatment using this modality and reduce financial burdens to society and chronic pain patients.

Data availability Underlying data
No data are associated with this article meta-analysis, but it also should be performed for NMA. In addition, how do the authors classify individual studies into high and low risk of bias?
There are now two Cochrane risk of bias instruments for RCTs. The authors propose to use the older one. It has advantages over the newer one but it has an important disadvantage: the unclear risk of bias categorization is problematic. It turns out that people can make reasonable inferences regarding risk of bias when it is unclear (see Akl et. al. 2012 4 ). The authors should consider using a revised Cochrane risk of bias instrument (see Guyatt GH, Busse JW. Modification of Cochrane Tool to assess risk of bias in randomized trials) 8.
If reviewers disagree on risk of bias, how will authors deal with the disagreement? 9.
Reviewer Expertise: Evidence-based medicine.
We confirm that we have read this submission and believe that we have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however we have significant reservations, as outlined above.
Author Response 10 Aug 2020 Laura O'Connor, National University of Ireland, Galway, Galway, Ireland Drs Ge and Guyatt, Thank you for this review of our protocol, we appreciate the time you have taken and your detailed feedback. Below, we respond to your specific points.
In the first view of the title, it appears the study aims to compare online and in person methods of psychotherapy, mainly focusing on the differences between delivered in person and online. However, in the methods section, we find that authors also compare different types of psychotherapies. Therefore, the authors should make a clearer study objective and more suitable title.

○
We have edited the title to highlight that the review intends to compare different psychotherapeutic techniques with one another, in addition to the modality by which these techniques are delivered.
The authors plan to use GRADE to assess the certainty (quality) of evidence. However, they do not refer to the latest advance of GRADE for NMA. We suggest that authors should carefully read the recent GRADE papers and make substantial revisions accordingly. In particular, they should plan to consider what they will do if they find sparse networks and the assumption of common heterogeneity leads to counterintuitive blowing up of the network estimate confidence intervals relative to those from the direct comparisons. Some references follow1-3.

○
We have edited this section to include the latest advancements in the GRADE criteria for NMAs based on the citations provided.
The authors should provide more detailed statistical analyses. First, how do the authors obtain direct, indirect, and NMA estimates? Second, how do the authors assess the transitivity? Thirdly, inconsistency assessment needs more details (if using GRADE they should probably use GRADE terminology, and use "incoherence" when they refer to differences between direct and indirect estimates). Finally, do they have any hypotheses regarding possible explanations of heterogeneity and if so what is the postulated direction of effect and are there any planned subgroup analysis or meta-regression to explore the source of heterogeneity?
○ We have added more detail into the analysis section describing how the direct, indirect and network estimates will be generated. We have also edited the language regarding the GRADE terminology. We have highlighted that we anticipate heterogeneity and made clear that subgroup analyses and meta-regressions will be considered in the presence of substantial heterogeneity. These analyses will also determine whether the assumption of transitivity will be met.

Minor suggestions:
Thank you again for your thoughtful feedback. This is an open access peer review report distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Miel A. P. Vugts
Tranzo Scientific Center for Care and Welfare, Tilburg School of Social and Behavioral Sciences, Tilburg University, Tilburg, The Netherlands The authors describe a network meta-analysis for determining which psychotherapy (approach) and mode of delivery (internet-based versus in-person) is most efficacious for patients with chronic pain. The question of how psychotherapy forms can be ranked in terms of efficacy is both relevant and complex. The protocol is clearly written down and PRISMA guidelines for systematic reviews and the extension for network meta-analyzes are followed. Limitations are well discussed. In my opinion, the article can be improved on several points. The writing style makes it easy to read. However, vague language is sometimes used. For example, "studies will be managed in Endnote".

○
The paragraphs about evidence for the efficacy of (online) psychotherapy largely focusses on statistical significance of previous studies. Maybe, a more nuanced description can be given, also mentioning the effect sizes and methodological quality of previous research and its implications: how about potential risks of bias assessed in previous reviews, and clinical relevance of psychotherapies for chronic pain (in general)?
○ Another potential area for improvement is the specific rationale for conducting a NMA: The statement "There has also been an increase in internet delivered interventions for chronic pain, however, there is a dearth of research evaluating the efficacy of online psychotherapies in comparison to their counterparts delivered in person. The current review will investigate these comparisons using a meta-analysis (NMA)" could also be read as a plea for more primary research on direct comparisons of psychotherapies via internet or in-person.

○
The objective has been formulated in terms of a method, comparing treatments is a means and not an end in itself. I suggest to formulate the objective in terms of a Explain a bit more about variation through 'intervention format or sample characteristics'. For example, patients with depression are sometimes included and sometimes not, the recruitment base can be 'open' (from the general public via internet) or 'closed' (from clinics or work settings), patients with particularly high education levels and computer self-efficacy to self-select for internet-delivered formats. Fidelity of implementation (not only attrition) can be an influence.
Is it possible to take account of some moderating factors in the NMA? Will there be enough information per type of psychotherapy to do so? 1.
Formulate clearer (theory-based) research questions and hypotheses (can you specify the "hypothesized values of the parameters"?). Since there is so much uncertainty about the processes by which psychotherapies are effective, I consider it to be legitimate to use NMA more as an exploratorative tool.

2.
More details could be added with respect to: -Inclusion and exclusion criteria: I suggest that you be more precise: Report clear criteria or exhaustive lists instead of examples of "psychotherapeutic content" and "debilitating diseases". This leaves questions like: is expressive writing instruction psychotherapeutic content? Is arthritis a debilitating disease?
The choice of another study to exclude patients with headache is not an actual argument to do the same. Maybe, add a paragraph with specific a priori classifications of how, and on what basis, different kinds of psychotherapy are based on approaches, techniques, and delivery modes. Are there any criteria with regard to the methods that studies have used to screen the patients? -Which systematic reviews will be searched to identify additional relevant studies? 3.
Describe how you will deal with risk of bias and small sample sizes of included studies in the NMA.

4.
Minor points: In "this review will use PICO to frame and report review criteria:", the second "review" could be replaced by "eligibility".
○ Add a reference to support: "However, when appropriately and conservatively used to synthesise research and aid decision making, they are extremely beneficial and influential". ○ "Cost-effectiveness": Is this really within the scope of this article? ○ To note, I have experience with conducting meta-analyses, but I have no expertise in NMA specifically.
Is the rationale for, and objectives of, the study clearly described? Partly

Is the study design appropriate for the research question?
Partly

Are sufficient details of the methods provided to allow replication by others? Partly
Are the datasets clearly presented in a useable and accessible format? Not applicable Thank you for taking the time to review our protocol, and for your very helpful feedback. We have followed your guidance and made several changes to the article, which we agree strengthens it. Below, we address your points in turn and indicate which specific changes we have made.
The writing style makes it easy to read. However, vague language is sometimes used. For example, "studies will be managed in Endnote".
We have edited the text to use clearer language and better explain specifics.
The paragraphs about evidence for the efficacy of (online) psychotherapy largely focusses on statistical significance of previous studies. Maybe, a more nuanced description can be given, also mentioning the effect sizes and methodological quality of previous research and its implications: how about potential risks of bias assessed in previous reviews, and clinical relevance of psychotherapies for chronic pain (in general)? Another potential area for improvement is the specific rationale for conducting a NMA: The statement "There has also been an increase in internet delivered interventions for chronic pain, however, there is a dearth of research evaluating the efficacy of online psychotherapies in comparison to their counterparts delivered in person. The current review will investigate these comparisons using a meta-analysis (NMA)" could also be read as a plea for more primary research on direct comparisons of psychotherapies via internet or in-person.
We have edited this section to state our rationale for carrying out an NMA more clearly, and separately from the need for more primary research on this topic. We have reformulated the objective to more explicitly state what the question that this NMA aims to answer. We intend to be clear about the need for more high-quality primary research as well as the need for syntheses like this in reporting our results, as we expect that findings based on indirect evidence will need to be supported by direct research before strong conclusions can be drawn. In terms of moderating factors, subgroup analyses and meta-regressions will be considered should substantial heterogeneity be evident.

1.
Formulate clearer (theory-based) research questions and hypotheses (can you specify the "hypothesized values of the parameters"?). Since there is so much uncertainty about the processes by which psychotherapies are effective, I consider it to be legitimate to use NMA more as an exploratorative tool. We have edited that section to highlight that the NMA is an exploratory analysis, where we are using the observed parameter estimates of the various treatments to generate comparisons without any priori hypotheses.

2.
More details could be added with respect to:-Inclusion and exclusion criteria:I suggest that you be more precise: We are constrained by the literature on this -it is likely that individual papers will describe their methods screening of participants in varying levels of detail, and with little consistency across papers as a whole. In many cases only screening criteria are covered, not the methods used to apply those criteria.
Which systematic reviews will be searched to identify additional relevant studies?Should systematic reviews be identified by the search that are eligible for inclusion, their references will be searched. 3.