{'fraction_non_alphanumeric': 0.02584675030962253, 'fraction_numerical': 0.013515696516073449, 'mean_word_length': 5.0142487046632125, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 5, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Normative values of the Brief Repeatable Battery of Neuropsychological Tests in a Brazilian population sample: discrete and regression–based norms

Objective: Cognitive dysfunction is common in multiple sclerosis. The Brief Repeatable Battery of Neuropsychological Tests (BRB–N) was developed to assess cognitive functions most–frequently impaired in multiple sclerosis. However, normative values are lacking in Brazil. Therefore, we aimed to provide continuous and discrete normative values for the BRB–N in a Brazilian population sample. Methods: We recruited 285 healthy individuals from the community at 10 Brazilian sites and applied the BRB–N version A in 237 participants and version B in 48 participants. Continuous norms were calculated with multiple–regression analysis. Results: Mean raw scores and the 5 th percentile for each neuropsychological measure are provided, stratified by age and educational level. Healthy participants’ raw scores were converted to scaled scores, which were regressed on age, sex and education, yielding equations that can be used to calculate predicted scores. Conclusion: Our normative data allow a more widespread use of the BRB–N in clinical practice and research.

The sample size was estimated for differences (or correlations) between two independent means by adopting an effect size of 0.5, α error probability of 0.05, and power (1-β error probability) of 0.80 for two groups (version A and B subgroups; two-tailed). In this way, the minimal sample size would be 128 individuals, divided into 64 for each subgroup. However, since most BRB-N normative studies in the USA and European countries employed at least 150 individuals for version A (the most used) 7,8 , we enrolled 237 participants for this version, and 48 participants for version B of the battery.
Inclusion criteria were ages from 18 through 65 years, educational level of at least primary school ( four years), visual acuity of at least 0.5 (or 20/40) in both eyes (with or without lens correction), normal neurological examination, and having slept enough the night prior to the testing. Individuals were excluded if they had neurological disease or major psychiatric illness, history of alcohol or drug abuse, serious head trauma, learning disability and a recent major medical illness. The study was approved by the ethics committee of the faculty of medical sciences of the University of Campinas and all participants provided written informed consent.

Neuropsychological test procedures
The BRB-N version A was applied in 237 participants and version B was applied in 48 participants. Different versions are important, to minimize practice effects with longitudinal administration. These groups were not different regarding gender distribution, age and educational level ( Table 1).
All neurologists and neuropsychologists who administered the battery had participated in an initial common training session in order to standardize the criteria of administration, data recording and scoring procedures. All individuals underwent testing during daytime and in a quiet room. The administration time of the BRB-N was around 30 minutes.

BRB-N subtests and scores
Verbal memory was tested with the Selective Reminding Test (SRT). The test consists of presenting orally to the individual a list of 12 unrelated words for up to six trials, at the rate of one word per two seconds. After the list has been presented, the participant is instructed to recall as many words as possible in any sequence. Words that are not recalled are repeated (reminded) by the examiner on the next trial, and the individual is requested to recall and say again the whole list of 12 words. The scoring system consists of a Long Term Storage (LTS) and a Consistent Long Term Retrieval (CLTR). If a word is recalled on two successive trials without a reminder, it is assumed to have entered LTS on the first of these two trials. With or without retrieval of this word on the subsequent trials, it is scored as LTS on all following trials. The CLTR score refers to consistent recall of this same word on all succeeding trials until the last one.
Cognitive function abnormalities are increasingly recognized as a major complaint among multiple sclerosis (MS) patients. Neuropsychological studies have shown cognitive dysfunction in up to two thirds of MS patients 1 , and it has been described in the earliest stages of a clinically, or even radiologically, isolated syndrome 2 . Cognitive dysfunction generally affects information processing speed and episodic memory and thus has a substantial contribution to disability, impairing daily living and work capacity. Nevertheless, assessing such an important and complex domain requires a systematic approach. Given the low sensitivity of the widely-used Mini Mental State Examination (MMSE) for detecting cognitive dysfunction in MS 3 , Rao and co-workers of the American National MS Society developed the Brief Repeatable Battery of Neuropsychological Tests (BRB-N), a battery of neuropsychological measures covering functions most commonly impaired in MS, with 71% sensitivity and 94% specificity 4 . This battery includes tests for the assessment of verbal and visuospatial memory, sustained attention and information processing speed, working memory and verbal fluency. More recently, an international panel introduced a shorter cognitive battery for use when time constraints are strict, comprising tests for information processing speed, verbal and visuospatial memory (the Brief International Cognitive Assessment for Multiple Sclerosis) 5 . Although the BRB-N has been employed extensively in MS research worldwide and is also validated in the Portuguese language 6 , normative values for this test battery have been published only in few European countries and the USA 7,8 , limiting a more widespread application in clinical practice, especially in South America 9 . The Brief International Cognitive Assessment for Multiple Sclerosis battery has also recently been validated in Brazil, but normative data is not yet available 10 . An accurate classification of neuropsychological impairment depends on the normative comparison. If normative data are derived from individual samples that do not match the individual under assessment, misclassification may occur. Therefore, development of appropriate normative standards is critical for any neuropsychological measure 11 . Neuropsychological normative data have usually been presented in terms of discrete norms, where means and standard deviations for each age group are provided. Nevertheless, discrete norms have been subject to criticism in the last few years and the use of regression-based demographically-adjusted scores has increasingly been recommended 11 . In this setting, our aim was to provide both discrete and continuous normative values for the BRB-N in a sample of the Brazilian population, while assessing the effect of demographic factors on cognitive performance.

Participants
We enrolled a total of 285 healthy individuals recruited from the community at 10 sites across Brazil (Belo Horizonte, After about 20 minutes, the subject tries to recall and replicate the list again (delayed recall) 12,13 . Versions A and B have a different list of 12 words. The scores employed are: LTS (total number of words in LTS in all six trials), CLTR (total number of words in CLTR in all six trials) and the number of correct words after delayed recall.
Visuospatial memory was tested with the 10/36 Spatial Recall Test (SpRT), which consists of a test where participants are shown, for a period of 10 seconds, a 6 x 6 checkerboard with 10 pieces placed in specific locations. Shortly afterwards, the participant is given 10 pieces and asked to replicate the pattern on a blank checkerboard. The test is repeated three times. After about 20 minutes, the participant tries to recall and replicate the pattern again (delayed recall) 13 . Versions A and B differ in the spatial configuration of the 10 pieces. There is an immediate recall score, equal to the total number of correct responses for the three trials, and a delayed recall score.
Sustained attention and information processing speed were tested with the Symbol Digit Modalities Test (SDMT). Only the SDMT oral version was employed in this study. The SDMT consists of a key with two rows, with nine stimulus symbols in the upper row and matched numbers (1-9) in the row below it. The task sequence consists of a series of symbols in random order, each with a blank space underneath, and the participants are asked to respond with the number that matches each symbol as rapidly as possible in 90 seconds, after completing a 10-item practice trial. The score of the test is the number of correct responses completed within the time limit, with the maximum score of 110 13,14 . Each alternate form (A and B) features a new arrangement of the symbol-digit pairings in the key (the same symbols matched in a different order).
Working memory and information processing speed were tested with the Paced Auditory Serial Addition Test. In this test, participants listen to an audio recording of 61 digits where single digits are presented every three seconds and the participant must add each new digit to the one immediately prior to it (and not to the result of the previous sum), providing each sum orally as quickly as possible. The test score is the number of correct sums given (out of 60 possible) in each trial. Versions A and B differ only in the sequence of digit presentation 13, 15 .
Verbal fluency was tested with the Word List Generation. This is a semantic verbal fluency test evaluating the spontaneous production of names of a given category (i.e. vegetables and fruits in version A; animals in version B) within 90 seconds. The score is the number of correct names in either category 13 . Inflections of the same word (cow, cows) or its perseverations are counted as one response, and words not belonging to the category are not counted.
Translation of the BRB-N stimuli and instructions was performed by three translators, fluent in both Portuguese and English. At first, a native English speaker translated from English to Portuguese. Then stimuli and instructions were back-translated into English and then into Portuguese, independently by two neurologists, both experts in neuropsychology and fluent in English, and members of the Neuropsychology and Neurolinguistics Unit of the University of Campinas. The final version was piloted in 10 healthy participants and 10 MS patients for final adaptation of sociocultural and linguistic aspects.

Statistical analysis
Statistical analysis was performed with the Statistical Package for the Social Sciences (v21.0, SPSS Inc, Chicago). The Continuous norms were calculated with multiple-regression analysis as previously described 11,16,17 . Briefly, healthy participants' raw scores on each neuropsychological measure were first converted to scaled scores based on cumulative distribution (M = 10, SD = 3). Scaled scores were then regressed on age, education and sex (male = 2; female = 1). We tested for multicollinearity among predictor variables. In addition, we inspected the distributions of residuals for normality by analyzing q-q plots. Plots of regression-standardized residual predicted values showed that variance of the residuals was constant (homoscedasticity). Given the small sample size of version B, norms (discrete and continuous) were calculated for version A only.

RESULTS
There were no significant differences of demographic variables between the participants in whom versions A or B were administered (Table 1). Mean scores were higher in version B for the SRT (LTS and CLTR) and SpRT (delayed recall only) ( Table 1).
Mean raw scores and the 5 th percentile for each neuropsychological measure in version A are shown in Table 2, stratified by age and educational level.
Healthy participants' raw scores in version A were converted to scaled scores (Table 3). These raw-to-scale-score conversions can be applied to MS patients. Scaled scores were regressed on age, sex and education yielding equations that can be used to calculate predicted scores (Table 4). Age was a significant predictor for all neuropsychological measures, except verbal fluency, while sex was the opposite.

DISCUSSION
Although a remarkable increase in cognitive-related MS research has been observed in the last years, there is still a considerable gap in applying this knowledge into clinical practice. This may be due to several reasons including absence of normative data, time restriction for the use of extensive neuropsychological batteries and lack of established therapeutic options to overcome cognitive    3,4 . Accordingly, we aimed to include a fairly significant number of individuals in the present study, representative of areas in Brazil where MS is more prevalent and where most MS centers are located, and we were able to provide normative data for the version A of the battery. The BRB-N has been widely used in MS research worldwide, including South America 7,8,9 . In Brazil, it was initially employed in the context of the CogniCIS, a multinational study evaluating cognitive performance in clinically isolated syndrome 18 . A further study by Brooks et al., has shown that the BRB-N is feasible for assessing cognition of MS patients in the daily clinic 19 . Moreover, this battery showed strong discriminating power between patients and controls in a Brazilian sample, insofar as large effect sizes were observed 20 . However, even though this battery had already been validated in the Portuguese language, Brazilian normative values were not yet available for these studies and analysis had to rely on adequate control groups 6 . Nevertheless, small control groups have potential limitations and one cannot adequately match all possible demographic variables.
We aimed to provide both discrete and continuous normative data. Discrete norms have some limitations and have been subject to criticism over the last years. For example, Morgan et al. have shown that the use of demographically-adjusted T-scores significantly improved sensitivity for discriminating impaired versus neurocognitively normal individuals in comparison to the raw cut score 21 . Similarly, Parmenter et al. demonstrated that using a continuous norm approach, higher rates of impairment were identified compared to standard norms for many of the minimal assessment of cognitive function in multiple sclerosis measures 17 , a neuropsychological battery developed with strong psychometrics properties for use in MS research, but with a considerably longer administration time compared to the BRB-N 1,22 . Recently, the shorter Brief International Cognitive Assessment for Multiple Sclerosis battery was validated in Brazil, but neither continuous nor discrete norms were available in our population 10 . Several authors have advocated the use of continuous norms based on multiple regression analysis accounting for demographic factors 11,16,17 . However, whether continuous norms are the most appropriate for answering all research and clinical questions is still a matter of discussion. For example, Silverberg and Millis showed that discrete norms may have worked better in determining whether the patients' cognitive abilities were sufficient for the demands of universal functional tasks, such as activities of daily living 23 .
Although, in our sample, raw scores obtained in all BRB-N measures were not normally distributed, we opted to also provide discrete norms stratified by age and education for ease of clinical use on a daily basis and to address selected questions. However, use of these norms should be made with the awareness of potential limitations, as discussed above and illustrated in the following case.
For example, consider a 32-year-old female MS patient with 11 years of education who scores 30 in the SDMT. According to discrete norms in Table 2, her score is within her age group mean-1 SD, and thus could be considered normal. However, her raw score corresponds to a scaled score of 6, according to Table 3 and her predicted scaled score on the SDMT is 10.15, based on the regression equation expressed in Table 4 [9.669 + 1(0.092) + 32(-0.090) + 11(0.297)]. We then divide the difference between her actual and predicted scaled score (6-10.15 =-4.15) by the standard deviation of the residual (2.343), obtaining a z score of-1.77. This value equals a T score of 32 and is considered impaired using an operational definition of "impairment" of either 35 or 40, as suggested by some authors 11,17 .
Regarding the BRB-N version B scores, we found them slightly higher only for the verbal (total learning) and visuospatial memory (delayed recall) tests when compared to version A and these findings were not attributable to gender, education or age differences. Although our sample size for the BRB-N version B test was relatively small, it is interesting that previous studies have found similar results for the SpRT (delayed recall) but not for the verbal memory test 7,8 . For example, Amato et al. have found higher scores in all BRB-N version B subtests except in the SRT total learning (CLTR and LTS) 8 . Nevertheless, in order to be used longitudinally with reliable results, both versions should be equivalent. Thus, even though the higher scores of the SRT version B may have been influenced by its smaller sample size, we need further studies comparing SRT versions A and B with equivalent words regarding length, frequency of use, and category membership. Subsequently, further research with more individuals performing version B will be able to address whether this version is suitable for longitudinal usage of the battery.
Also in line with previous results, we found that demographic characteristics, and especially age and education, were significantly related to all neuropsychological measures, except the Paced Auditory Serial Addition Test, where there was a trend for only sex and age 7,8 . Sex was only associated with the Word List Generation score. These findings reinforce the need for adjusting raw scores for demographic variables when interpreting neuropsychological test scores.
In conclusion, our normative data allow a more widespread use of the BRB-N in clinical practice and research, providing both discrete and continuous norms. However, discrete norms should be used with caution, and demographically-adjusted scores are generally preferable when interpreting neuropsychological test scores.