{'fraction_non_alphanumeric': 0.030046511627906978, 'fraction_numerical': 0.014, 'mean_word_length': 5.1909012381226605, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 4, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 14, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Measuring early childhood development in Brazil: validation of the Caregiver Reported Early Development Instruments (CREDI)

Objective The present study aims to analyze the psychometric properties and general validity of the Caregiver Reported Early Development Instruments (CREDI) short form for the population-level assessment of early childhood development for Brazilian children under age 3. Method The study analyzed the acceptability, test-retest reliability, internal consistency and discriminant validity of the CREDI short-form tool. The study also analyzed the concurrent validity of the CREDI with a direct observational measure (Inter-American Development Bank's Regional Project on Child Development Indicators; PRIDI). The full sample includes 1,265 Brazilian caregivers of children from 0 to 35 months (678 of which comprising an in-person sample and 587 an online sample). Results Results from qualitative interviews suggest overall high rates of acceptability. Most of the items showed adequate test-retest reliability, with an average agreement of 84%. Cronbach's alpha suggested adequate internal consistency/inter-item reliability (α > 0.80) for the CREDI within each of the six age groups (0–5, 6–11, 12–17, 18–23, 24–29 and 30–35 months of age). Multivariate analyses of construct validity showed that a significant proportion of the variance in CREDI scores could be explained by child gender and family characteristics, most importantly caregiver-reported cognitive stimulation in the home (p < 0.0001). Regarding concurrent validity, scores on the CREDI were significantly correlated with overall PRIDI scores within the in-person sample at r = 0.46 (p < 0.001). Conclusions The results suggested that the CREDI short form is a valid, reliable, and acceptable measure of early childhood development for children under the age of 3 years in Brazil.


Introduction
A strong foundation in early development is a prerequisite for individual health and well-being, as well as harmonious societies. 1 The importance of early childhood development (ECD) is well reflected in the Sustainable Development Goals (SDGs), which mandate access to early care and educational opportunities for young children around the world by 2030. 2 Investments in ECD services are particularly necessary in low-and middle-income countries, where the proportion of children who are not reaching their developmental potential remains high. 2,3 Consistent monitoring of ECD needs and outcomes using culturally and developmentally appropriate measures is a key to ensuring the success of interventions. 4,5 Population measures of ECD encourage a focus on children's abilities in multiple domains, and can be used to compare large groups of children and to track overall progress of population of children. 6 These assessments can leverage culturally relevant, evidence-based ECD policies and targeted investments to improve the potential of a nation's young children. 7 In recent years, measures of children's ECD status have been developed for large-scale use, including the UNICEF's Early Childhood Development Index (ECDI), which assesses children aged 36---59 months, 8 and the Inter-American Development Bank's Regional Project on Child Development Indicators (PRIDI), which evaluates children aged 2 to almost 5 years through direct observation. 9 Much less information is currently available on the development of children under the age of 3, among whom direct assessments tend to be more limited in scope and are generally harder to implement.
In this context, we examine the short form of the Caregiver Reported Early Development Instruments (CREDI; available in Supplementary Material Appendix A), which was developed as a new caregiver-reported instrument for assessing the overall development of children under the age of 3. 10 The aim of the CREDI short form is to provide conceptually rich, developmentally informed, population-level data on global progress in alleviating ECD-related inequities and meeting target 4.2 of the SDGs. 10

The Brazilian context
Brazil's recent early childhood legislation (Marco Legal da Primeira Infância) establishes principles and guidelines for public policies, including programs that educate families to stimulate children's development. 11 This legislation highlights that public childhood programs need to involve monitoring and systematic data collection and the dissemination of these evaluation results. 11 However, neither needs assessments nor impact evaluations will be feasible without adequate instruments to assess child development.
Previous studies have reviewed tools used in Brazil to assess ECD and screen for developmental difficulties. 12---14 The most commonly used and cited tools are the Bayley Scales of Infant Development (BSID) and the Denver Developmental Screening Test. An adapted version of the Ages and Stages Questionnaire (ASQ) has also been used in Brazil. 15 Despite the utility of these individual-level assessments, several limitations have been identified for the large-scale use of these tools, including the relatively high cost associated with application kits, test administration fees, materials, and highly trained professionals. 16 Another challenge is that most of the instruments used in Brazil were developed in high-income countries, and have not yet been culturally validated in other contexts. 13,14 Instruments that assess population-level development in a cheap and scalable fashion are not currently available in Brazil, which makes comparisons within and across countries currently impossible.

The present study
In the present study, we aim to analyze the validity of the CREDI short form for the population-level assessment of ECD for Brazilian children under the age of 3. To do so, we analyze the acceptability, test-retest reliability, internal consistency, construct validity, and concurrent validity of the CREDI.

Study sample and procedures
The study includes two samples: a sample of children from São Paulo (southeast Brazil) previously enrolled in an intervention study and interviewed in-person, and an online sample with participants from different parts of Brazil. The in-person sample comprises 678 children aged 28---35 months, and the online sample includes 587 children aged 0---35 months ( Table 1).
All children in the in-person sample are part of the Western Region Birth Cohort, which includes all children born between October 2013 and March 2014 at the University Hospital of São Paulo. These children represent approximately 80% of all children in the public health system of the area and are primarily from low-and middle-income families living in São Paulo's western region. A total of 900 caregivers were randomly selected from the larger cohort and agreed to participate in the PRIDI assessment. These 900 children do not differ from the rest of the cohort with respect to any observable characteristics. From these, 678 mother-child dyads completed the study including all child development assessments. The online sample was recruited through a Facebook group run by a Brazilian pediatrician that provides pediatric health and wellness information (e.g. healthy eating and disease prevention). A total of 1265 caregivers expressed interest in the study by clicking on the Facebook link. Of these, 587 caregivers completed all sections of the online survey and were thus included in this study. Participants who reported their geographical information (n = 523; 89%) were from five regions of Brazil (Southeast, 66%; South, 16%; Northeast, 12%; Midwest, 5%; and North, 1%). Mothers in this group were on average substantially more educated than the Brazilian average.
A smaller sample of 38 caregivers was recruited from the in-person sample in São Paulo to take part in brief cognitive interviews designed to assess understanding and appropriateness of the CREDI items within the Brazilian cultural setting. This subsample was recruited according to mothers' availability and was stratified by children's gender and age, ensuring representation of boys/girls and older/younger children. Demographic characteristics for these caregivers were generally similar to those of the broader São Paulo cohort.

Ethical considerations
The study was reviewed by the institutional review board (IRB) at the Harvard Graduate School of Education. The São Paulo data collection was done as part of protocol number 890.325 approved by the University Hospital of Universidade de São Paulo's (HU-USP) IRB. All caregivers were informed about the objectives of the study and provided informed consent prior to answering the study's questions.

Measures CREDI
The CREDI is an internationally developed, population-level measure for assessing the overall development of children aged 0---35 months across the motor, language, cognition, socioemotional, and mental health domains. This tool was designed to be usable within large-sample data collection efforts, and to be culturally neutral with items that are not affected by culturally specific contexts. 10 This opensource tool can be downloaded freely from the CREDI website (https://sites.sph.harvard.edu/credi/). The CREDI is administered directly to the child's primary caregiver using a yes/no response scale. There are two versions of the CREDI. The short form (which is the focus of the present study) creates a summary score for children's overall developmental status, whereas the long form creates domain-specific developmental scores. The short form includes 20 items specific to each six-month age group (0---5 months, 6---11 months, 12---17 months, 18---23 months, 24---29 months, and 30---35 months), and the administration time is on average five minutes. The CREDI is scored continuously using age-standardized scoring procedures that are based on children's raw percent ''yes'' (pass) responses within each age group. The short form was originally developed based on an extensive multi-stage, multi-country pilot effort that included both quantitative and qualitative data analysis focusing on the items' psychometric properties and cultural and developmental appropriateness. 10 In the present paper, we specifically focus on the performance of the CREDI short form items in Brazil. All items were translated to and from Brazilian Portuguese by native speakers.

PRIDI
A direct assessment tool of children aged 24---59 months, including 21 items for capturing four domains of ECD: cognition, communication and language, socioemotional, and motor. 9 For the present study, the PRIDI was only administered to children aged 2---3 years in the São Paulo in-person cohort.

Household stimulation
Caregivers in both samples reported on cognitive stimulation using items from UNICEFs Multiple Indicator Cluster Survey ECD module capturing adult-child interactions in six different activities (e.g. reading, telling stories, and playing), over the preceding three days. Stimulation scores represent the total number of activities endorsed by caregivers (range = 0---6), with higher scores indicating more stimulation.

Asset quintile (1---5)
For the in-person sample, we followed a methodology to classify participating households into wealth quintiles. 17 Principal component analysis of the following variables was conducted: household ownership of a motorbike or car, number of bathrooms in the household, as well as child ownership of picture books, bed, and separate bedroom.
In the online sample, all respondents were directly asked to assess their income relative to others on a scale from 0 to 100, with 0 meaning poorer than everyone else, 50 meaning average, and 100 meaning higher income than everyone else. This assessment was based on similar measures of relative socioeconomic status (SES). 18 This information was used to divide the sample into quintiles.

Data analysis
The CREDIs acceptability in Brazil was assessed using 38 qualitative interviews conducted by a trained Brazilian data collector with families living in São Paulo: 18 interviews focused on items in the cognitive and language domains and 20 focused on items in the socioemotional and mental health domains. Each caregiver responded to the CREDI item and then was asked to discuss in her/his own words the meaning of the question. Two independent coders (including the data collector and a CREDI team member) rated the caregivers' understanding of the item as either matching with the original item intent (1) or not (0). When coders disagreed, a third CREDI team member served as a tiebreaker. Items were deemed to be well understood if at least 80% of caregivers received a score of 1.
To analyze test-retest reliability, 120 caregivers within the in-person sample from São Paulo were interviewed using   the CREDI twice over the course of approximately ten days. Kappa statistics were computed to assess the alignment of responses between the two interviews. Additionally, overall agreement (percentage of caregivers providing the same answer) for each item was calculated. Cronbach's alpha was computed for each of the six age groups to assess the internal consistency of the CREDI. Construct validity was assessed using a hypothesistesting method. 19 The in-person and online samples were assessed using separate linear regression models examining score differentials with respect to child and family characteristics, including child age, gender, stunting status (only for the in-person sample), household stimulation, SES, and maternal education levels. Based on prior ECD research, our hypothesis was that children who were female, had caregivers with higher education and SES, and came from high-stimulation households would demonstrate higher CREDI scores. Concurrent criterion validity was assessed by correlating scores from the CREDI with scores from the PRIDI direct assessments conducted as part of the in-person interviews in older children in São Paulo. Associations between the CREDI and the PRIDI were compared within subgroups based on caregiver education level as an initial step in testing for invariance. All analyses were conducted using the Stata statistical software program (version 14).

Results
Qualitative interviews revealed an overall high acceptability of the scale, as well as high degrees of cognitive understanding of items. Most items were clearly understood by more than 80% of caregivers. One socioemotional item demonstrated 75% understanding, and two cognitive items demonstrated 75% and 67% understanding, respectively. No issues were detected with any of the items, and the participants were cooperative with and felt pleased by the items. As such, and given that these same items demonstrated greater than 80% understanding across countries, all items were retained at this stage. Results indicated that most of the items showed adequate test-retest reliability, with an average agreement of 84%, and a minimum agreement of 75% across all age groups ( Table 2). In terms of kappa, five items showed excellent reliability (kappa > 0.80), 32 items showed substantial reliability (kappa > 0.60), and 15 items showed moderate reliability (kappa > 0.40). Ten items, most of the socioemotional domain, showed fair to low reliability (≤0.40).
The CREDI scores were moderately correlated with overall PRIDI scores; conditional on age, we found a correlation coefficient of r = 0.46 (p < 0.001) in our in-person sample. Fig. 1A shows a local polynomial curve of normalized PRIDI scores as a function age-normalized CREDI scores along with 95% confidence intervals. Fig. 1B shows the same empirical association between direct observation scores (PRIDI) and CREDI scores by caregiver educational attainment. No statistically significant differences in the observed correlations were found across the three strata of interest (primary, secondary, higher education), suggesting initial evidence for measurement invariance across socioeconomic groups.

Discussion
The CREDI was found to be a highly acceptable tool by both the in-person and online Brazilian samples of caregivers. In both samples, the caregivers did not display any difficulties answering CREDI questions. The qualitative interviews confirmed these findings, with high rates of cognitive understanding across items. The instrument also showed adequate internal consistency across the six age groups. However, the age bands 0---5 months and 18---23 months had few participants and thus need to be further investigated.
Test---retest reliability was moderate to excellent for most of the items, and the rates of agreement were consistently high. The items that showed a low kappa were the most within the socioemotional domain and tended to represent behaviors that are potentially less stable over time and context (e.g. kindness to other children). These findings are consistent with the literature that argues that low kappa values may not necessarily reflect low rates of overall agreement. 20 Nevertheless, these findings indicate a need for further exploration of the stability and reliability of caregivers' reports, particularly in terms of young children's socioemotional skills.
Regarding the construct validity, the findings from the inperson sample showed that children receiving higher CREDI scores tended to be female, have caregivers with higher education, and come from households that were higher in socioeconomic status and stimulation, as expected. In the more geographically diverse and demographically advantaged online sample, on the other hand, the only robust predictor of CREDI scores was the households' stimulation levels. One reason for the lower levels of discrimination in this sample could be that the sample was more homogenous in terms of education and wealth than the in-person sample, as caregivers were recruited through social media and willing to participate in a written online survey. It is also possible that the more subjective measure of wealth used in this sample introduced error into the estimation, masking true differences. Therefore, the online sample was on average significantly wealthier and better educated than caregivers in the São Paulo sample. However, the home stimulation scores were relatively similar across both samples.
CREDI scores in the in-person sample of children aged 2---3 years also showed adequate concurrent criterion validity with the PRIDI, which uses direct observation of the child to assess early development. These results suggest that caregivers' reports using a shorter instrument correspond well to a similar population-level assessment from Latin America that uses a more detailed format. Similarly to the CREDI, prior research using the PRIDI has shown that a nurturing environment is associated with child development. 9 Collectively, these findings support the hypothesis that interventions targeting positive caregiver-child interactions may be effective in closing gaps in child development. Parenting programs that focus on child development and caregiver---child interactions have been shown to be effective within Brazilian samples, highlighting their relevance for future scaling-up. 21 Importantly, stimulation practices explained only a relatively small amount of variation in CREDI scores. Given this, comprehensive and multi-faceted programs that directly target children's health, nutrition, and early education are needed alongside programs for families to optimize children's outcomes. 22 This basic principle is reflected in Brazil's ''Marco Legal da Primeira Infância'' legislation. 11 The CREDI could therefore be an option for monitoring long-term progress toward this goal, as well as evaluating intervention programs to support child development at a population level.
Furthermore, the CREDI may also be used as a potential indicator for tracking progress toward meeting SDG 4.2. Existing population-level measures of ECD (e.g. ECDI) 8 tend to focus on older children only. The CREDI ---which was designed explicitly to ''bridge'' with the ECDI through a set of common items ---may therefore serve as a complementary measure of ECD status for the youngest, and potentially most vulnerable children.
Despite the strengths of this study, there are also some limitations that must be addressed through future work. First, our focus on a single geographic context for the inperson sample and the use of a convenience sample in the online survey sample substantially limit the generalizability of these results. Second, it was not possible to use the same socioeconomic measure for the in-person and online samples, precluding direct comparisons between these groups. Finally, the concurrent validity, with direct observation, was performed only for children aged 2---3 years. Future studies should include samples from geographically, linguistically, developmentally, and culturally diverse contexts of Brazil; should utilize alternative approaches to establishing construct validity (e.g. factor analysis); should use similar measures for socioeconomic level; should examine concurrent validity with samples from 0 to 2 years old; and should include the CREDI as an outcome measure in the context of intervention evaluation.
In conclusion, the results of the present study suggest the CREDI short-form's validity, reliability, and acceptability as a measure of ECD within Brazil. These findings encourage the use of this instrument for large-scale surveys and monitoring efforts of early developmental outcomes in Brazilian children under the age of 3.