{'fraction_non_alphanumeric': 0.02358887952822241, 'fraction_numerical': 0.008658616493494337, 'mean_word_length': 5.538249694002448, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Using GRADE as a framework to guide research on the sexual and reproductive health and rights (SRHR) of women living with HIV – methodological opportunities and challenges

ABSTRACT In March 2016, WHO reviewed evidence to develop global recommendations on the sexual and reproductive health and rights (SRHR) of women living with HIV. Systematic reviews and a global survey of women living with HIV informed the guideline development decision-making process. New recommendations covered abortion, Caesarean section, safe disclosure, and empowerment and self-efficacy interventions. Identification of key research gaps is part of the WHO guidelines development process, but consistent methods to do so are lacking. Our method aimed to ensure consistency and comprised the systematic application of a framework based on GRADE (Grading of Recommendations, Assessment, Development and Evaluation) to the process. The framework incorporates the strength and quality rating of recommendations and the priorities reported by women in the survey to inform research prioritisation. For each gap, we also articulated: (1) the most appropriate and robust study design to answer the question; (2) alternative pragmatic designs if the ideal design is not feasible; and (3) the methodological challenges facing researchers through identifying potential biases. We found 12 research gaps and identified five appropriate study designs to address the related questions: (1) Cross-sectional surveys; (2) Qualitative interview-driven studies; (3) Registries; (4) Randomised controlled trials; and (5) Medical record audit. Methodological challenges included selection, recruitment, misclassification, measurement and contextual biases, and confounding. In conclusion, a framework based on GRADE can provide a systematic approach to identifying research gaps from a WHO guideline. Incorporation of the priorities of women living with HIV into the framework systematically ensures that women living with HIV can shape future policy decisions affecting their lives. Implementation science and participatory research are appropriate over-arching approaches to enhance uptake of interventions and to ensure inclusion of women living with HIV at all stages of the research process.


Introduction
Development of normative guidance is a key role of the World Health Organization (WHO); such guidance informs the development and revision of national policies and guidelines as well as the development of programmes. The process of WHO guideline development includes identification and documentation of gaps in the available evidence base, notably in the balance of benefits and harms of an intervention (World Health Organization, 2014). Identification of these gaps is intended to guide focused research to inform and strengthen future WHO recommendations.
At a three-day meeting in March 2016, WHO formulated new evidence-based recommendations on sexual and reproductive health and rights (SRHR) of women living with the human immunodeficiency virus (HIV). The consolidated guideline brings together relevant, existing recommendations from WHO guidelines, and also includes new recommendations that were developed with the help of a Guideline Development Group (GDG) of experts, including people living with HIV. The GDG reviewed evidence from systematic reviews in order to formulate new consensus-based clinical recommendations and good practice statements (World Health Organization, 2017). The clinical recommendations were specific to interventions not previously addressed in other WHO guidelines and covered the following four topics: Caesarean section, mode of abortion, safer disclosure of HIV serostatus, and interventions to enhance self-efficacy and empowerment. The good practice statements included guidance on psychosocial support, sexuality, economic empowerment and resource access (including food security), and integrated service delivery, and were not further included in this study.
Research gaps were proposed by the GDG informed by the evidence from systematic reviews and from a global survey of the SRHR priorities of women living with HIV. The survey was commissioned by WHO specifically for this process to be led by and for women living with HIV (Narasimhan et al., 2016;Salamander Trust, 2014). The community survey was led by a global core group of 14 women living with HIV and comprised an online, solution-focused survey of 832 women living with HIV from 94 countries, supplemented by focus groups with 113 women living with HIV from seven countries.
In this article, we present a systematic approach to the prioritisation of the research questions identified during the guideline process, identify optimal study designs, and propose alternative designs and strategies to overcome practical or methodological challenges which may arise. We expanded the methods used in a prior WHO guideline and employed the Grading ofRecommendations, Assessment, Development and Evaluation (GRADE) approach as a framework to guide this process (Siegfried, Beanland, Ford, & Mayer, 2015). We aimed to demonstrate the feasibility and utility of using a structured process within a WHO guideline context to identify gaps in the research of SRHR of women living with HIV.

The GRADE approach and WHO
WHO has adopted the GRADE system for the development of global clinical, programmatic and public health guideline recommendations (Guyatt et al., 2008;World Health Organization, 2014). GRADE provides guideline developers with a transparent, systematic approach to aid their decision-making . Each recommendation is formulated in response to an a priori clinical or programmatic question regarding the effectiveness of an intervention. Questions are structured following the PICO format (specifying the Population, Intervention, Comparator, and Outcomes of interest for the review) and are drafted by a WHOled Steering Group in advance of the final GDG meeting. A systematic review is conducted for each question prior to the GDG meeting and the overall quality of evidence is rated as high, moderate, low or very low, dependent on the risk of bias, precision, consistency, and directness of the results. WHO recommendations are then formulated by the GDG in response to the quality of the evidence combined with an assessment of benefits and harms, resource utilisation, user values and preferences, feasibility, ethics, equity and human rights (World Health Organization, 2014). These domains are summarised in an Evidence to Decision Table (See Table 1). Recommendations are categorised by the GDG as strong or conditional. In general, strong recommendations are made when the quality of evidence is high and the benefits of an intervention outweigh the harms, whereas conditional recommendations recognise that the quality of the evidence is low or that specific country contextual factors will determine the uptake of a recommendation (Andrews, Schunemann et al., 2013). Table 1. Key domains that require consideration when formulating WHO recommendations.

Factor
How the factor influences the direction and strength of a recommendation Quality of the evidence The quality of the evidence across outcomes critical to decision-making will inform the strength of the recommendation. The higher the quality of the evidence, the greater the likelihood of a strong recommendation.

Values and preferences
This describes the relative importance assigned to health outcomes by those affected by them; how such importance varies within and across populations; and whether this importance or variability is surrounded by uncertainty. The less uncertainty or variability there is about the values and preferences of people experiencing the critical or important outcomes, the greater the likelihood of a strong recommendation Balance of benefits versus harms This requires an evaluation of the absolute effects of both benefits and harms (or downsides) of the intervention and their importance. The greater the net benefit or net harm associated with an intervention or exposure, the greater the likelihood of a strong recommendation in favour or against the intervention Resource implications This pertains to how resource-intense an intervention is, whether it is cost-effective and whether it offers any incremental benefit. The more advantageous or clearly disadvantageous the resource implications are, the greater the likelihood of a strong recommendation either for or against the intervention Priority The problem's priority is determined by its importance and frequency (i.e., burden of disease, disease prevalence or baseline risk). The greater the importance of the problem, the greater the likelihood of a strong recommendation.

Equity and human rights
The greater the likelihood that the intervention will reduce inequities, improve equity or contribute to the realisation of one or several human rights as defined under the international legal framework, the greater the likelihood of a strong recommendation Acceptability The greater the acceptability of an option to all or most stakeholders, the greater the likelihood of a strong recommendation. Feasibility The greater the feasibility of an option from the standpoint of all or most stakeholders, the greater the likelihood of a strong recommendation. Feasibility overlaps with values and preferences, resource considerations, existing infrastructures, equity, cultural norms, legal frameworks, and many other considerations. Note: Reproduced from the World Health Organization (2014).

Methods
We adopted a similar approach to future research formulation used in a previously published WHO guideline for HIV post-exposure prophylaxis (Siegfried et al., 2015).
The GRADE system provides a framework to determine research gaps based on the strength of the recommendation and quality of the evidence with conditional recommendations and low quality evidence clearly identifying when further research is required. We expanded upon this to include a component specific to the values and preferences of the community the recommendations intend to servein this case, women living with HIV. Following the GDG meeting, we tabulated the following for each a priori PICO question, and for additional research questions identified from the global survey: (1) The clinical, programmatic or values-based research question identified by the GDG or the survey; (2) The related recommendation formulated by the GDG or a record that no recommendation was made or that the question was not part of the current GDG process; (3) The strength of each recommendation where applicable; and (4) The quality of evidence underpinning the recommendation where applicable From the above, a research gap was identified when one or more of the following conditions was met: (1) the recommendation was conditional; or (2) the quality of evidence was low or very low (even in the presence of a strong recommendation); or (3) no recommendation was made and the GDG or survey had articulated a research gap. The authors then considered and tabulated the following: (5) The most appropriate and most robust study design to answer the future research question(s); (6) Conceptualisation of a pragmatic alternative study design if the ideal design was not feasible; and (7) Consideration of the methodological challenges of the study design with the potential bias(es) identified.

Results
Table 2 (See Supplemental data) summarises the 12 research gaps identified specifically around the four main PICO topic areas newly covered in the guidelines: Caesarean section, mode of abortion, safer disclosure of HIV serostatus, and interventions to enhance self-efficacy and empowerment. Five distinct study design formats were identified as feasible and appropriate to address the 12 research questions: (1) Cross-sectional surveys (2) Qualitative interview-driven studies; (3) Establishment of a global registry, or national registries; (4) Randomised controlled trials of comparative effectiveness; and (5) Medical record audit.
We identified selection bias as the primary methodological challenge in mixed methods studies incorporating cross-sectional surveys and qualitative key informant interviews. Healthcare providers and women who choose to participate in such studies may exhibit similar characteristics and be qualitatively different from those who choose not to participate, limiting the generalizability of the results. In some circumstances this may be desirable such as when exploring the promotion of healthy sexuality of women living with HIV as women who have considered this issue will have rich experiences and observations to share which can guide the development of health promotion tools.
Additional methodological challenges identified included confounding in cross-sectional surveys and registries limiting causal inference, and measurement bias arising when blinding is not possible in RCTs or when evaluation tools are not validated to measure highly sensitive outcomes in surveys. Registries are dependent on the quality of data collected and as such are susceptible to misclassification bias, especially if women choose not to disclose their HIV status or do not agree to anonymised testing. Contextual bias is an important consideration in generalising from RCTs of assisted partner notification or disclosure strategies as culture and context will likely play a significant role in the effectiveness of such strategies.

Discussion
Application of a GRADE framework to a WHO guideline on the SRHR of women living with HIV enabled the systematic identification of 12 questions for future research. Our framework comprises the strength of relevant WHO recommendations, the quality of the current supporting evidence base, and a strong values and preferences component to ensure SRHR priorities and desires of women living with HIV are included directly into the research prioritisation process. Five study designs and associated methodological challenges were identified for each of the 12 research questions.
Randomised controlled trials remain the gold standard for evaluating effectiveness where equipoise exists and provide the highest quality evidence for formulating WHO recommendations. However, several of the identified research questions require evaluation of complex, multi-faceted, and often multi-sectoral interventions which may be better suited to evaluation within an implementation research paradigm. Implementation research seeks to explore effects in real-world conditions, mindful of context and user needs (Peters, Adam, Alonge, Agyepong, & Tran, 2013). Outcomes can include acceptability, feasibility and costs in addition to conventional measures of effectiveness. Collaboration with those who will implement and receive the intervention in the design and conduct of the research is a recognised strength of implementation research (Peters et al., 2013). As a paradigm it fits well with the desire of women living with HIV to be equal participants in the research endeavour as articulated in the global survey (Narasimhan et al., 2016).
The global survey of women living with HIV demonstrated that many women continue to experience stigmatising attitudes and behaviours within healthcare settings and from many cadres of healthcare providers when accessing sexual and reproductive health or HIV care (Salamander Trust, 2014). This was noted to be experienced across diverse legal and policy environments. A 2015 systematic review identified an absence of peerreviewed studies of implementation of rights in healthcare programming and called for urgent research to better integrate rights into healthcare interventions (Kumar, Gruskin, Khosla, & Narasimhan, 2015). To ensure provision of rights-based care and better uptake of services, community participation is an essential element in the success of SRHR and HIV programmes. Negative behaviours of healthcare providers need to be clearly identified and described in future research methodologies, aiming: (1) to provide supportive evidence of the need to implement awareness-raising and training regarding appropriate caring and non-discriminatory behaviours among healthcare providers; (2) to guide development of focused interventions and accountability mechanisms; and (3) to ensure community participation in order to reduce community-based barriers to service uptake and retention in care. Research methods include surveys supplemented with participatory structured qualitative interviews which can be shaped, delivered and analysed by peers. Where the human rights of women living with HIV are shown to be infringed, participatory research using quantitative and qualitative methods may also be appropriate to further explore the role of, and need for, accountability mechanisms within the healthcare sector (Jagosh et al., 2015;Raab & Stuppert, 2014). Participatory research processes may be unfamiliar to many scientists. They take time, funds and expertise in participatory methodologies. However they are increasingly recognised as critical to understanding the psychosocial challenges of patient-oriented care (Jagosh et al., 2015;Peters et al., 2013;Salamander Trust, 2014).
In addition to the time-limited and participantdependent study designs outlined during the prioritisation process, the team also identified two passive means of data collection and analysis: medical audit and establishment of registries. In situations where active data collection is expensive or not feasible, both methods can provide useful indications of association and trends providing data integrity is ensured, and suitable adjustment is made for potential confounding during the analysis stage. For example, a global termination of pregnancy (TOP) registry would shed light on whether reasons for, and consequences of, TOP differ for women living with HIV. Databases of TOP exist in several countries and provide insight into the scale, reasons, methods, and outcomes of TOP (Justad-Berg, Eskild, & Strom-Roum, 2015;McCall, Flett, Okpo, & Bhattacharya, 2016;Svensson et al., 2014). Monitoring of TOP indications and outcomes would allow for evaluation of accountability mechanisms for ensuring that TOP services are equitably offered regardless of HIV status. Importantly, a TOP registry can enumerate and explore reasons for repeat TOP which may, for example, indicate a failure of effective contraceptive services and/ or inaccurate guidance from healthcare providers regarding the SRHR of women living with HIV. Understanding and addressing ineffective contraception services was raised by the survey participants as a priority for women living with HIV with a call for women to be at the centre of all decision-making regarding service provision which affects their lives (Narasimhan et al., 2016;Salamander Trust, 2014). Newer models of registry development adopt a patient-led approach and align data collection with patient priorities (Nelson et al., 2016). Women with HIV should ideally be included in the design, oversight and use of a future TOP registry.
Previous researchers have collated and reviewed the evidence base in the field of SHRH of women living with HIV and have identified a comprehensive set of research gaps (Kumar et al., 2015;Loutfy, Sonnenberg-Schwan, Margolese, & Sherr, 2013). Our paper is limited in scope as the gaps identified are specific to the new recommendations included in the consolidated WHO guideline, and are not intended to be a comprehensive future research agenda for all SRHR issues of women living with HIV. However, our method expands prior research prioritisation approaches as we involved women living with HIV throughout the process and incorporated their voices directly into the GRADE framework.
Transparency is a key strength of the GRADE approach which has here been expanded to identify research gaps based on the strength of the recommendation and the quality of the evidence rating. However, reliance on the strength and the rating of quality alone is insufficient for guiding future research as gaps may remain even when recommendations are strong or quality is high. Paradoxically there are instances when the quality of evidence is low, but it is acceptable to make strong recommendations, such as in life-threatening situations (Alexander et al., 2014). In these circumstances, while more research may be desirable, it may be ethically impossible to conduct more studies. Guideline developers should therefore continue to apply their critical judgement when applying the GRADE framework to identify research gaps.

Conclusions
Use of the modified GRADE framework to establish a focused research agenda for SRHR of women living with HIV is feasible and contributes towards the goal of WHO to collaborate with those most affected by guidelines (World Health Organization, 2014). A systematic approach to identifying research gaps and the necessary study designs to address these provides researchers, including women living with HIV, and funders with the necessary information to conduct appropriate, high-quality research required for future revisions of the guidelines. Adoption of participatory and implementation research will ensure maximal involvement of women living with HIV at all stages of the research process and ultimately, in SRHR decision-making which directly affects their lives.