{'fraction_non_alphanumeric': 0.028368118428158454, 'fraction_numerical': 0.009053654817497378, 'mean_word_length': 5.408794788273616, 'pattern_counts': {'":': 0, '<': 1, '<?xml version=': 0, '>': 0, 'https://': 1, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Working Group Recommendations for the Practice of Teleneuropsychology in Latin America

Abstract Objective Teleneuropsychology (teleNP) could potentially expand access to services for patients who are confined, have limited personal access to healthcare, or live in remote areas. The emergence of the COVID-19 pandemic has significantly increased the use of teleNP for cognitive assessments. The main objective of these recommendations is to identify which procedures can be potentially best adapted to the practice of teleNP in Latin America, and thereby facilitate professional decision-making in the region. Method Steps taken to develop these recommendations included (1) formation of an international working group with representatives from 12 Latin American countries; (2) assessment of rationale, scope, and objectives; (3) formulation of clinical questions; (4) evidence search and selection; (5) evaluation of existing evidence and summary; and (6) formulation of recommendations. Levels of evidence were graded following the Oxford Centre for Evidence-Based Medicine system. Databases examined included PubMed, WHO-IRIS, WHO and PAHO-IRIS, Índice Bibliográfico Español en Ciencias de la Salud (IBCS), and LILACS. Results Working group members reviewed 18,400 titles and 422 abstracts and identified 19 articles meeting the criteria for level of evidence, categorization, and elaboration of recommendations. The vast majority of the literature included teleNP tests in the English language. The working group proposed a series of recommendations that can be potentially best adapted to the practice of teleNP in Latin America. Conclusions There is currently sufficient evidence to support the use of videoconferencing technology for remote neuropsychological assessments. These recommendations will likely contribute to the advancement of teleNP research and practice in the region.


Introduction
Neuropsychological assessment is critical for the detection of cognitive impairment and the evaluation of cognitive functioning. Patients are most often referred for differential diagnosis and characterization of their cognitive profile, an important element for rehabilitation, treatment planning, and crafting forensic reports (Vakil, 2012).
Standard neuropsychological assessment methods rely on face-to-face interactions, which are not always possible due to a variety of factors, such as distance and isolation. Although access to healthcare is a problem for people in rural areas worldwide, the aforementioned barriers are especially relevant in developing countries. The Latin American region is characterized by unequal access to healthcare, with most resources and facilities concentrated in large cities, limiting access to specialized practitioners for small, scattered, and isolated towns and villages. Commonly encountered barriers to healthcare access are centered upon transport and communication (Strasser, 2003). According to the Pan American Health Organization (PAHO), 21% of the region's population is deterred from seeking care due to geographic barriers (Organización Panamericana de la Salud, 2014). Furthermore, developing countries do not have an adequate number of healthcare professionals, which, in combination with long distances, has further hindered access to healthcare during the COVID-19 pandemic (Ham-Chande & Nava-Bolaños, 2019).
Teleneuropsychology (teleNP) has become a valuable tool to overcome these barriers. Telemedicine provides an alternative to face-to-face practice that allows isolated and remote rural populations to access quality healthcare at a reasonable cost (Morgan et al., 2011, Calandri et al., 2021. From an economic perspective, telemedicine has been shown to reduce healthcare costs and be less time-consuming than face-to-face approaches (Castanho et al., 2016).
The emergence of the COVID-19 pandemic has significantly accelerated the use of teleNP (Bloem, Dorsey, & Okun, 2020;Fox-Fuller, in press). Despite the implementations of varying levels of lockdown, the need to continue to care for vulnerable patients has significantly intensified efforts to develop remote cognitive assessment protocols and adapt evaluation tools and tests. Similar to other parts of the world, the number of neuropsychologists in Latin America using teleNP in their daily practice has exponentially increased (Willis Towers Watson, 2021). Latin America covers 13% of the world's total land surface area and shares many critical aspects of culture; with a population of 653,962,331 individuals, 56,222,679 of which are over the age of 65 (WHO & The World Bank, 2017), the projections for the number of elderly individuals in the population are increasing, and rates of dementia are rising (United Nations, 2017). Diagnosis, follow-up, and interventions for dementia have been affected during the COVID-19 pandemic, creating a challenging situation for neuropsychologists in Latin America. However, there is little research on the use of teleNP in Latin America.
The present recommendations aim to support standardized clinical procedures that will incentivize data generation in our region. These clinical practice recommendations aim to assess and summarize the best available evidence to help Latin American health professionals and endorse strategies to approach patients using remote platforms, which will facilitate decision-making in daily practice.
These recommendations follow those of the Appraisal of Guidelines for Research and Evaluation instrument (AGREE II) (Brouwers, Kerkvliet, & Spithoff, 2016) and were constructed in the following stages: 1. assessment of the justification, scope, and objectives; 2. selection of the working group; 3. formulation of clinical questions; 4. evidence search and selection; 5. evidence evaluation and summary; and 6. formulation of recommendations.

Formation of the Working Group
An open call for Latin American experts on neuropsychology, practicing teleNP was made at the "Tele-Neuropsychology in Latin America" webinar organized by the Ibero-American Forum of Neurosciences on June 17, 2020. The working group

Justification, Scope, and Objectives
The COVID-19 pandemic has underscored the lack of an adequate framework for regulating teleNP practices in Latin America. This study aims to establish a consensus on recommendations regarding ethical, clinical, and practical aspects of conducting cognitive assessments using teleNP.
For these recommendations, teleNP is defined as the provision of neuropsychological services using telecommunication technology, specifically videoconferencing. The scope of these recommendations will include recommendations for neuropsychologists who perform cognitive assessments remotely using different videoconference platforms, which are not intended for patients, their families, or professionals outside Latin America. Furthermore, the recommendations are exclusively aimed to guide direct-to-home "full-teleNP" types of assessment and not other assessment models (e.g., the trained technician "teleinterview" model; the assistant-proctored "full-teleNP" model; or the hybrid in-clinic and direct-to-home "teleNP" model) (Stolwyk, Hammers, Harder, & Munro Cullum, 2020). The recommendations will focus on cognitive assessment issues only and will not include recommendations for cognitive rehabilitation.

Formulation of Clinical Questions
The group of experts formulated a series of important questions for the practice of teleNP in Latin America. Each member was instructed to develop five relevant questions that (a) noted a clinical care need in Latin America, (b) contributed to the standardization of clinical practice, and (c) proved essential to improving quality of care in Latin America. The final question selection was made by consensus in an assembly with all representatives from the Latin American working group for teleNP (see Table 1). These questions were answered based on scientific evidence from which the recommendations were formulated.

Evidence Search and Selection
The following databases were accessed for the literature review: PubMed, WHO-IRIS, WHO and PAHO-IRIS, IBCS (Índice Bibliográfico Español en Ciencias de la Salud), and LILACS, a repository of scientific and technical literature on health produced by Latin American and Caribbean authors and published in Latin American and Caribbean countries started in 1982. Searches were conducted with the following terms alone and in combination: "teleNP," "telemedicine," "videoconference," "Telehealth," "Telemedicine," "teleNP," "Teleneuropsychological Assessment," and "Teleconference-Based Neuropsychological Assessment." To identify recent literature, only evidence published during the last 10 years (from October 30, 2009 to November 10, 2020) was included. Articles including general guidelines and recommendations suitable for Latin America were adopted. Articles describing tests previously applied using teleNP in Hispanic and Latin American populations were reviewed. A total of 28,203 articles were identified, and 9,803 duplicates extracted from the databases were removed. A total of 18,400 titles were reviewed by 10 experts from the panel to evaluate the degree of fit within the scope of the recommendations. Each examiner received a total of 3,680 titles for review; in this manner, each title was reviewed by two blinded examiners. Titles were included only when both examiners agreed on inclusion. Consequently, 422 titles were selected for abstract review, and only articles that included adult subjects and that were written in English, Spanish, or Portuguese were included, further reducing the number of eligible articles to 76. These 76 articles were reviewed by the whole expert panel, and 19 were selected because they suited the scope of these recommendations (see Fig. 1). It is important to note that while articles describing the use of teleNP in Hispanic and Latin American populations were searched, the majority of the resulting articles mainly described studies conducted in the United States and in English.
Levels of evidence were established according to the classification proposed by the Oxford Centre for Evidence-Based Medicine (CEBM) (see Tables 2 and 3) (Heneghan, 2009). This classification is continuously updated and available at www.ce bm.net/levels_of_evidence.asp. The degree of recommendation was assessed following the system suggested by Guyatt et al. (1995). Individual cohort study (including low quality RCTs; e.g., <80% follow-up) 2c Outcomes research; ecological studies 3a SR (with homogeneity * ) of case-control studies 3b Individual case-control study 4 Case series (and poor-quality cohort and case-control studies) 5 Expert opinion without explicit critical appraisal or based on physiology, bench research or "first principles" SR = systematic review; RCT = randomized clinical trial. Higher-quality studies conducted on this topic arrived at conflicting conclusions. The recommendation is based on these conflicting studies E Theoretical/foundational evidence A preponderance of evidence from animal or cadaver studies, from conceptual models/principles, or from basic sciences/bench research support this conclusion F Expert opinion Best practice based on the clinical experience of the guidelines development team

Evidence Evaluation and Summary
Of the 19 studies reviewed, 14 used a case-control design, 3 were systematic reviews, and 2 were expert group opinions on the subject. Data from a total of 733 subjects were analyzed. The subjects' mean age, calculated from the studies that presented such data, was 67.42 years old. Regarding the condition of the subjects, six studies showed data from patients with dementia or major neurocognitive disorders, five from patients with mild neurocognitive disorder, seven from healthy controls, two from individuals in a psychiatric population (alcohol abuse and schizophrenia), and one from individuals experiencing traumatic brain injury sequelae. Only one study also reported results for caregivers. Table 4 summarizes the evidence evaluated and grades of recommendation assigned.

Identifying Barriers in Latin America
To achieve our main goal of creating a set of recommendations that could address regional challenges and barriers to teleNP, we asked the panel of experts to mention at least five challenges facing Latin American teleneuropsychological practice.
After gathering the results of this survey, we clustered the responses according to frequency and returned to the panel with a consolidated list of the most prevalent obstacles in the region. Experts were then asked to agree upon possible solutions for these barriers.

Population
Cognitive assessment by video conferencing is a useful tool for adult patients (18-90 years old) with mild or major neurocognitive disorder (or dementia) and for cognitively healthy individuals. Evidence indicates that patients with visual or auditory deficits, acute confusional states, or severe communication difficulties should not be evaluated using teleNP  No studies were found that included patients with less than 12 years of education (high school diploma). Therefore, no recommendations can be made on the use of teleNP in populations with low education levels. Education level may correlate very well with technological literacy (Kämpfen & Maurer, 2018), which means poorer performance can be expected with remote assessments in the low-education level group.

Informed Consent
Before receiving psychological services of any kind, patients must give informed consent and sign the appropriate form (Class F Recommendation). Recommendations for types of informed consent were drawn from multiple sources, including the American Psychological Association (APA) and the InterOrganizational Practice Committee recommendations/guidance for teleNP in response to the COVID-19 pandemic .
Consent should address the concerns relevant to services provided using language reasonably understandable to a layperson. It is recommended that cultural, linguistic, and other issues that may impact the subject's understanding be considered. Consent should include unique concerns arising from the teleNP assessment framework: scope of the intervention, confidentiality, use of personal data, whether the interview will be recorded, and screenshot management. (Class F Recommendation) .
Informed consent should include some key points: Written or oral consent. The use of teleNP implies that written consent may be difficult to obtain. For this reason, the use of alternative approaches approved by law in each country is suggested. In this regard, options such as recorded verbal consent or an electronic signature can be used. Regarding digital consent, the use of a system in which information access, user identity, and security can be guaranteed is strongly recommended (Class F Recommendation) (Latin American working group for teleneuropsychology).
There are multiple platforms where e-consent could be registered. Due to its availability, lack of cost, and proven data safety, the use of REDCap is recommended for this purpose (Class F Recommendation) (Latin American working group for teleneuropsychology).
Privacy. It should be made clear that the use of teleNP may carry additional risks to privacy and confidentiality (Class F Recommendation) (Joint Task Force for the Development of Telepsychology Guidelines for Psychologists, 2013).
Furthermore, it should be explicitly stated that the patient must not take screenshots of the teleNP session, and the neuropsychologist will take screenshots only with previous notice and with the intention of capturing visual material that has been drawn or written by the patient (Class F Recommendation) (Latin American working group for teleneuropsychology).
Results. Most tests are adaptations of the original face-to-face versions. It should be clarified that some qualitative data normally collected from face-to-face exams are no longer acquired, which may limit recommendations and conclusions (Class F Recommendation) .
Local law. Consent must be obtained within the framework of the legal provisions enforced by each country. It is essential to know the applicable relevant laws and the regionality of these laws. As a general rule, when practices take place across countries and there is a conflict between the laws of the two countries, the laws of the patient's location prevail over the laws of the neuropsychologist's location (Class F Recommendation) (Sports Neurpsychology Society, 2020).

Strategies for Conducting a TeleNP Assessment
The APA has published a checklist with strategies for conducting psychotherapy via telehealth in a Telepsychology Checklist (https://www.apa.org/practice/programs/dmhi/research-information/telepsychological-services-checklist). The InterOrganizational Practice Committee has modified and adapted those guidelines and identified some key points for guidance in conducting teleNP assessments. We present a summary of the most relevant critical points for Latin American populations: Before a teleNP assessment.
• Screen and review referral information by reviewing the patient's medical history to ensure that the patient is suitable for teleNP assessment. TeleNP is not appropriate for all patients (see Population). • Ensure that the patient has adequate equipment and software for teleNP assessment. An email can be sent to the patient or the facilitator with technological requirements for the interview (platform, type of connection, screen size, use of headphones). • State the need for a private and silent environment that is free from distractions. This can be established in a previous communication and be repeated immediately before starting the assessment. • Decide if the patient will require an on-site facilitator; if so, define the facilitator's role in advance of the session. The patient's lack of familiarity with electronic devices and the videoconference platform may justify the presence of a facilitator.
• Clarify that only the patient and the on-site facilitator, if required, should be present in the room. The aforementioned facilitator can be either a family member or a professional. Collateral information can be obtained from either the family member or the facilitator. • Send the informed consent form and received a signed copy.
During the teleNP assessment.
• Disable the recording option and state that session recording is not allowed (this should be included in the informed consent form). • Confirm the patient's identity and establish an emergency plan in case the connection is interrupted. The emergency plan should consist of establishing an alternate form of contact with the patient at the beginning of the interview (e.g., a mobile phone of the patient or the family member/facilitator). If the connection is lost, an attempt should be made to reestablish it; if this is not possible, another signal source should be used, such as a mobile instead of a broadband signal. If the professional considers that the latter is not an option, the assessment should be resumed as soon as possible (ideally within the same day). This should also be noted in the report. Tests that are interrupted should be administered from the beginning when contact with the patient is resumed. It should also be noted that for those tests that present with a higher risk of learning, an alternate version should be administered. • Check that the patient has the necessary materials for the assessment (pen, pencil, paper). • Keep a written copy of all protocol material. If possible, keep a written register of interruptions and distractions during the session. • Use the "share screen" function to show the stimuli. • Be careful when the stimuli must be kept out of patient sight (for example, in visual memory tests). In these cases, describe the situation to the patient or explain this to the facilitator. It is useful to describe this at the beginning of the session: patients may thus obtain a folder or paper to cover the stimuli. • Screenshots can be obtained by the examiner, with previous notice, only when needed to capture and record evidence of tasks that require the patient to write or draw. In contrast, the patient is prohibited from doing so, especially in tests that assess memory. This should be included in the informed consent form.

Presence of an On-site Facilitator
A facilitator's presence is not always required for proper test administration. (Class B Recommendation) (Brearly et al., 2017). However, there is evidence that a facilitator's presence during the administration of tests with a motor component ensures greater validity regarding the use of these tests. For this reason, the presence of a facilitator is recommended when administering tests involving motor responses. (Class B Recommendation) (Temple, Drummond, Valiquette, & Jozsvai, 2010;Turkstra, Quinn-Padron, Johnson, Workinger, & Antoniotti, 2012).
Although the panel did not determine specific recommendations for patients with under 12 years of education, the experts suggested that on-site facilitators may be especially relevant for patients with a low education level and low familiarity with the technology (Class F Recommendation) (Latin American working group for teleneuropsychology).

Technological Requirements
A broadband connection is recommended to reduce the likelihood of sound or image disruption or failure (Class B Recommendation) (Brearly et al., 2017;Castanho, Sousa, & Santos, 2017).
Video transmissions are severely affected by bandwidth. It is essential to test the connection on the practitioners' side, and it is recommended that broadband internet be used during the assessment. If possible, the platform should adapt to a changing bandwidth without losing the connection (American Telemedicine Association et al., 2014;Postal et al., 2020). Technical problems may compromise the validity of the teleNP assessment. If the transmission quality is low, relevant test information may be lost, and patient behavior may be altered. Given failures in sound and video, the person administering the test may need to repeat the instructions. Furthermore, the patient must engage with the assessment from a screen that is at least 13 in size, which excludes smartphones (Class F Recommendation) (Latin American working group for teleneuropsychology).  (Brearly et al., 2017).
Motor dependent tasks. The variability in the results from published studies on motor-dependent tasks suggests the need for further research on the use of these tests in teleNP. Current evidence suggests that the administration of high-motor component tests with on-site support/assistance yields comparable results (Class B Recommendation) (Temple et al., 2010;Turkstra et al., 2012).
Recommendations for the use of specific tests. Specific tests that fall within these recommendations and the associated levels of recommendation are included in Table 5.
Use of normative data. The studies and recommendations reported in Table 5 compare face-to-face to videoconference assessments. In these studies, learning effects were controlled using a randomized order on the administration form. Scores obtained in both modalities were compared. Before providing psychological services of any kind, patients must give and sign an informed consent. F Consent must be obtained within the framework of the legal provisions enforced by each country. Recorded verbal informed consent or consent through electronic signature can be obtained, but its validity depends on local laws. For digital consent, the use of a system which guarantees information access, user identity and security is recommended. The REDCap platform is recommended due to its availability, open access and security. Consent should include unique concerns arising from the teleNP assessment framework: scope of the intervention, confidentiality, use of personal data, whether the interview will be recorded and screenshot management. It should be made clear that the use of teleNP may carry additional risks to privacy and confidentiality. It should be clarified that most of the conclusions are based on face-to-face assessments, which limits conclusions and recommendations of the teleassessment. Strategies for conducting a teleNP assessment Before the assessment: F Check patient profile for suitability for a teleNP assessment. Ensure that the patient has adequate equipment and software for a teleNP. Secure a private and silent environment. Clarify that only the patient and the on-site facilitator, if required, should be present in the room. Send and receive signed informed consent. During the assessment: F Confirm the patient's identity and establish an emergency plan in case the connection is interrupted. Ensure that the patient has the necessary materials for the assessment. Keep a written copy of all protocol material. Use the "share screen" function to show stimuli. Control when stimuli must be out of patient sight. Obtain screenshots only when needed to capture and record evidence of tasks that require the patient to write or draw. Presence of facilitator A facilitator's presence is not always required for proper test administration. B A facilitator's presence is recommended when administering tests with a motor response. B On-site facilitators may be especially relevant for patients with low education and low familiarity with technology.

F Technological requirements
A broadband connection is recommended to reduce the likelihood of sound or image disruption or failure.

B
Patients must use a screen that is at least 13 in size, which excludes smartphones. F Selection of tests and use of normative data The use of verbally dependent tasks is recommended. B The use of tests with motor responses requires on-site support/assistance. B Use of normative data For selected studies (Table 5), normative data from face-to-face tests may be used. B Face-to-face and remote administration of the recommended tests yielded values that did not significantly differ between the two (see Table 5) and showed significant correlations, which indicates the possibility of using normative data obtained from face-to-face assessments (Class B Recommendation) (Cullum et al., 2014;Marra, Hamlet, Bauer, & Bowers, 2020).
Regarding the use of norms, it is important to clarify that the use of face-to-face norms in neuropsychology should also follow the general principles of neuropsychology, such as using age, sex, and educational level-paired norms if possible. Additionally, although these recommendations are mostly based on an English bibliography, normative data from language-appropriate resources must be used. Table 6 depicts a summary of the recommendations.  Table 7 includes a list of barriers and their consequent recommendations, which emerged from the responses to a survey distributed to the Latin American working group for teleneuropsychology. The list is ordered according to barrier frequency. All recommendations are Class D.

Limitations
The limitations of this paper are mostly related to the diversity of cultures, laws, and populations within Latin America due to the great number of countries and the vast territory covered by the region. Although Latin American countries share languages and certain cultural traits, each community also has its own distinctive features, which is why these recommendations do not go into specific details but are intended to initiate collaborations that allow the identification of some general aspects of teleNP in Latin America. We promote and support the design of research that strengthens these clinical practices with clinical evidence.
Another significant limitation of our work is that the studies from which we based our recommendations were mostly written in the English language. This is due to the lack of studies on Spanish-speaking populations from Latin America (Arias et al., 2020). However, studies on Spanish-speaking populations in the United States and in Spain support the applicability of teleNP in Spanish (Franco, Bernardo-Ramos, & Soto-Pérez, 2012). Providing recommendations for a structured framework for teleNP could be the first step in gathering Latin American clinical data. We these recommendations will require updating in the near future, as this is a first attempt to organize and standardize emergent practices related to teleNP in Latin America and therefore to incentivize the creation of regional evidence.

Conclusions
There is sufficient evidence to support the use of video conferencing technology for remotely administering neuropsychological tests. TeleNP assessments can be administered to patients with cognitive impairment but should not be administered when there is a history of visual or auditory deficits, severe major neurocognitive disorder, acute confusional states, or significant communication difficulties. Furthermore, no recommendations can be made on the use of teleNP in populations with low education levels, as there is insufficient literature on the subject.
Prior to implementation, informed consent describing the characteristics of the teleNP assessment must be obtained, and key practical considerations that can help prepare for and organize the teleNP assessment should be discussed. Regarding test selection, those requiring verbal responses from participants are recommended. Further research is necessary to determine the validity of tests that require motor responses. However, the current literature supports the notion that these tests also yield reliable results comparable to their face-to-face counterparts if administered with a facilitator. In the absence of normative data from video conferencing, available studies recommend using norms obtained in face-to-face assessments.
The Latin American working group for teleneuropsychology addressed particularly problematic aspects of teleNP practices in Latin America. The lack of evidence to overcome these barriers underscores the need for research on teleNP in the region. The recommendations we provided to overcome some of the barriers can serve as useful guides for clinical practice, but they are also limited in scope.
These recommendations are provided as a first step in our efforts to create a framework and encourage Latin American clinical research on teleNP, which we hope can aid to generate more culturally appropriate guidelines, and specific to different regions within Latin America. Furthermore, given the diversity and heterogeneity of the Latin American region, we encourage local groups working with teleNP to adapt these recommendations, taking into account their specific challenges when assessing their populations.
These recommendations are a response to the immense demand imposed by the COVID-19 pandemic in Latin America. Future studies should include counterbalanced crossover studies from Latin America, exploring whether scores acquired using teleNP differ from those acquired using on-site assessment modalities.
Providing Latin America with teleNP recommendations is an important step to unify the work currently in progress and to encourage the use of teleNP in the region. The Latin American working group for teleNP was established to encourage reliable practices in the region. Finally, we remind professionals of the ethical importance of completing training in the technical, legal, and clinical areas of assessments using videoconferencing before adopting remote assessment practices.