{'fraction_non_alphanumeric': 0.03146407519515692, 'fraction_numerical': 0.014696511072168234, 'mean_word_length': 5.173838209982788, 'pattern_counts': {'":': 0, '<': 8, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
The Reliability and Validity of the Invalidating Childhood Environment Scale (ICES) – Turkish Version

The reliability and validity of the Invalidating Childhood Environment Scale (ICES) – Turkish Version Objective: The Invalidating Childhood Environment Scale (ICES) was developed in order to assess the childhood experiences of invalidation retrospectively. As there is no tool to assess the invalidating environment for Turkish samples, the current study aimed to examine the psychometric properties of the scale and to adapt it to the Turkish language. Method: The English scale was first translated into Turkish and then a back translation was made into the original language. Three hundred and three university students participated in the study. Along with the ICES, Short Form of the EMBU (Egna Minnen av Barndoms Uppfostran [My Memories of Upbringing]) and the Rosenberg Self-Esteem Scale (RSES) were also administered. Results: According to the results of confirmatory factor analysis, it was seen that the values obtained were acceptable for both forms. It was found that the ICES mother form has positive and low associations with EMBU mother overprotection, EMBU mother rejection, and RSES. The ICES mother form was found to be highly and negatively associated with the EMBU mother warmth subscale. The same correlations as for the mother form were found in the ICES father form. According to the results of the current study, Cronbach’s alpha coefficients of ICES for mother and father forms were found to be 0.84 and 0.87, respectively. Conclusion: The findings of the current study show that the Turkish version of the ICES has acceptable levels of reliability and validity.


INTRODUCTION
I n the development of emotional and behavioral problems, the role of family and parents always occupies an important place as an etiological factor. While some studies report relations between parenting styles, methods of discipline, and styles of communication and children's adjustment or psychological signs (2), there are a number of studies focusing on a retrospective analysis of parent-child relationships for the assessment of adult psychopathology levels (3). Since the 1990s, some of these studies have drawn attention to the relation between an 'invalidating environment during childhood' and adult psychopathology (4,5).
The concept of 'invalidating environment during childhood', included in Linehan's biosocial model (6), refers to invalidation of a child's emotional experiences by their parents; punishment, ignorance or devaluation of expressions of certain emotions or their utterance. As a result, the child (a) learns to invalidate emotions, (b) experiences serious difficulties to recognize and organize emotions, and (c) applies maladaptive ways to cope with the discomfort it feels (7).
A similar approach is found in Kohut's model of the psychology of the self. According to him, the self develops when the baby and later the child internalizes the experiences with idealized and mirroring self-objects (8,9). During the early period of development, processes of the self, such as selfpacifying, reaching a feeling of security and completeness, being taught to regulate one's feelings, and a sustainable development of self-esteem, are driven by the mother as the baby's self-objects (10). If the mother's approach to the child is characterized by empathy, perceiving the requirements of the baby with whom she is bonding adequately and giving appropriate responses, the baby can obtain the necessary experiences to develop a feeling of self, and thanks to this empathetic approach the mother can take on a mirror function and make sure that the baby perceives the experiences of the self-objects and her emotional states as if they were its own (10). This mirror function, a process at the basis of the baby's development of self, is known simply as mirroring (11). In the absence of the self-objects empathy, mirroring, because of deficiencies in internalization and dissociation processes, does not complete the development of self in the required way and leads to disorders of the self. Kohut (11) classed the disorders of the self under five headings: psychosis, schizoid, paranoid, narcissistic personality disorder, borderline states, and narcissistic behavior disorder.
Linehan (6) explained the effects of an invalidating environment under four main topics according to this, the first outcome of an invalidating environment is the failure of the child to learn how to recognize and control his/her emotional reactions adequately, because his/her emotional expressions has been invalidated; it is expected to control his/her emotions without having been taught, and finally the problems of the emotionally fragile child are being ignored and no efforts made to solve these issues. The second problem of an invalidating environment consists in an exaggerated simplification of solutions to possible problems, which does not teach the child how to confront his/her discomfort or how to create realistic goals and expectations. The third result is the creation of exaggerated or extreme emotional reactions. In an invalidating environment, the child's reactions can only be answered when he/she gives extreme emotional responses or displays over-extreme problems. Punishing negative emotions and encouraging exaggerated/extreme emotions inconsistently makes the child vacillate between emotional inhibition and extreme emotional states. Finally, an invalidating environment fails to teach the child to feel confident about the validity of his/her emotional and cognitive reactions to personal or situational events. Instead, it teaches the child not to validate his/her experiences actively and thus to organize his/her behaviors, thoughts, and emotions according to cues taken from the environment. Consequently, invalidated persons will either quit their environment or change their behaviors in a way that allows them to be accepted in their social environment.
A growing body of research suggest that there are personal and interpersonal negative consequences of invalidation. Both internalizing symptoms like depression, anxiety, social avoidance; and externalizing symptoms like impulsivity, rule breaking and agression ,which are also seen in antisocial personality and psychopathology, are found to be related with invalidation (12,13). In addition, the experience of invalidation has been found to be related with interpersonal difficulties such as difficulties in emotion regulation, instability in romantic relationships and impairments in functioning (14).
Mountford et al. (1) develeoped an instrument in order to assess the invalidation experienced during childhood which is called Invalidating Childhood Environment Scale (ICES). This scale, developed during a research aimed to assess the associations between symptoms of eating disorders and invalidating childhood environment and distress tolerenace, is a retrospective self-report instrument. The limited number of studies which used this scale mostly examined the concept of invalitadion in relation to personality disorders (15)(16)(17)(18) and eating disorders (19)(20)(21).
In the related lietrature there are also other scales measuring a similar construct to Linehan's concept invalidation experiences (22), such as the Parental Acceptance Rejection Questionnaire (23). The fact that the present scale consists of fewer items than the other instruments measuring similar constructs can be considered as an advantage of the scale. Furthermore, Linehan's model suggests that, an invalidating environment, in the developing child, can create the perception that his/her experience is not accurate. It was reported that, this perception leads to problems in the development of emotion regulation skills like, the capacity to tolerate emotional distress or the ability to name his/her emotion (6). There are plenty of studies showing that emotion regulation diffucilties are related especially with eating disorders and borderline personality disorder (24)(25)(26)(27). It can be said that the usage of the current scale, especially in studies regarding the specified psychopathologies, will provide oppurtunity to compare the results with the other studies.
Main aim of this study is to determine the validity and reliability of the Turkish ICES form in a sample consisting of university students. Within the scope of the current study, internal consistency coefficients of the scales were calculated via Chronbach's alpha coefficient, while factor structure was assesed with confirmatory factor analysis. In addition, in order to assess the criterion-related validity, correlations between invalidation during childhood and measures of perceived parenting styles and selfesteem, which are thought to be related with invalidation, were examined.

Rosenberg Self-Esteem Scale (RSES):
This instrument developed by Rosenberg (28) consists of 63 items and 12 subscales. The Self-Esteem subscale, one of the 12 subscales, is a scale consisting of 10 items scored on a four-point Likert-type scale (1: strongly agree -4: strongly disagree). Five of the items (numbers 3, 5, 8, 9, and 10) are reverse-scored. Lower scores from this scale indicate higher levels of self-esteem. Reliability and validity study of Turkish version was conducted by Cuhadaroglu (29) and Chornbach's alpha coefficient was reported as 0.75. Chronbach's alpha internal consistency coefficient was found as 0.86 in the current study. assessing parents' attitudes separately for mother and father. The scale consists of three dimensions: overprotection, rejection, and emotional warmth. For the attitudes of mother as well as father, the dimension emotional warmth consists of 7 items, overprotection of 9 items and rejection of 7 items. Only the 17 th item in this instrument is scored in reverse as in the original study. In another study, Dirik et al. (32), assessing the psychometric properties of the instrument, reported Cronbach's alpha values of 0.79, 0.73, and 0.71 for fathers' emotional warmth, overprotection, and rejection, and 0.75, 0.72, and 0.64 for mothers' emotional warmth, overprotection, and rejection, respectively. In the current study Chronbach's alpha coefficients were 0.86, 0.80, and 0.78 for fathers' emotional warmth, overprotection, and rejection, and 0.84, 0.79, and 0.76, respectively, for mothers' emotional warmth, overprotection, and rejection.

Invalidating Childhood Environment Scale (ICES):
The ICES is a retrospective scale investigating the experience of invalidation by parents during childhood. It consists of two distinct sections. The first one includes 14 items scored on a 5-point Likert-type scale (1: never -5: always) assessing mother and father separately. For each parent, the score ranges between 14 and 70. In the second section, 4 items addressing to the family types identified by Linehan (6) are entered. Of the family types defined by these items, 3 are invalidating (typical, perfect, and chaotic), and these family types cause low discomfort tolerance. Typical invalidating family environments are found in families focusing on success and achievements, requiring emotion control. Another invalidating type, the perfect family, expects to hide emotions and behave as if they do not exist, without any tolerance for negative emotions. Finally, in the chaotic invalidating family type the parents may have alcohol or substance problems and be generally unavailable for their children. In these kinds of families, children commonly need to find solutions on their own. The fourth family type defined by Linehan (6) is one where the family provides adequate support and encouragement for their children's emotions. This is the only family type defined as supportive/validating rather than pathological, with no invalidating environment characteristics. Furthermore, these families teach their children how to identify their emotions and other internal experiences and to value these feelings. The second section of the scale asks the participants to score the degree of similarity of the family types described in each item to their own family on a 5-point scale. Robertson et al. (15) assessed the psychometric properties of the instrument and reported Cronbach's alpha values of 0.88 for the mother's form and 0.90 for the father's.

Procedure
Before starting the study, approval from the Ethics Committee for Social Sciences of Mersin University was obtained. Before translation, permission was received via electronic mail from the responsible author of the original instrument. With the author's permission and guidance, the scale was translated from the original English into Turkish and subsequently backtranslated from Turkish to English and reconfirmed by the corresponding author. In order to make sure that the scale was understood by the participants, a pilot study was carried out with 20 persons of the same profile as the target group before starting the actual study. The pilot study was performed on a one-to-one basis, timing the administration and giving the participants the opportunity to ask about items they did not understand. The definitive form of the instrument was determined at the end of the administration after the participants confirmed that there were no incomprehensible items. Thereupon, the data collection from the target sample began. Before administering the scales, participants were given assurance that their participation was voluntary and their personal data would not be used. No inclusion criteria other than being a university student and a volunteer were applied.
Administering the instruments took around 20 minutes. Data sets from 328 persons were entered into a statistics package. Data from 25 participants who had either not completed certain forms or apparently filled them in randomly were removed from the set. Before running the data analysis, the Mahalanobis distance was calculated, showing that the data set did not include any outliers. Empty data in the set were filled in using the mean value of the respective group. Eventually, analyses were run with the data from 303 participants.

Statistical Analysis
In this study, statistical methods in line with the criteria suggested by Gungor's (33) study on the development and adaptation of measurement instruments in psychology was used. Cronbach's alpha reliability coefficient was calculated for the internal consistency of the scale, and the split-half reliability was assessed using Guttman Split-Half analysis. To assess the construct validity of ICES, confirmatory factor analysis was carried out. The criterion-dependent validity of the scale was assessed using correlation analysis controlled for relations with the s-EMBU subscales and the RSES. The reliability coefficients of the scale and the correlations between them were calculated using the SPSS 20.0 package, while confirmatory factor analysis was carried out with the LISREL 8.51 package.

Confirmatory Factor Analysis for the Invalidating Childhood Environment Scale (Turkish)
In contrast to traditional factor analyses, confirmatory factor analysis is used to confirm a factorial structure previously determined by the researcher (34). In order to examine the degree of similarity between the covariance matrix and the dependent matrix of the variables measured in the study, specific fit indices were used. Goodness of fit index (GFI), comparative fit index (CFI), adjusted goodness of fit index (AGFI) and root mean square error of approximation (RMSEA) were taken into account as part of the confirmatory factor analysis. An acceptable range for the RMSEA value is 0.08-0.10, while values above 0.10 are not acceptable (35). The AGFI ranges between 0 and 1, and approaching 1 indicates an improved fit. A value of 0.90 for this index shows a good fit, also values above 0.85 are accepted as good fit (36). CFI values between 0.95 and 1 are a criterion for perfect fit, while values between 0.90 and 0.95 have been reported as acceptable (37,38). The GFI statistics ranges between 0 and 1, and approaching 1 indicates a better fit (36). The fit index is obtained dividing the χ 2 value by the degrees of freedom; values below 3 indicates a good fit (39).
The analysis aimed to confirm a single-factor structure of the ICES Mother form resulted not in a perfect fit, but the values were within acceptable limits ( χ 2 [71, N=303]=181.29, χ 2 /df=2.55, RMSEA=0.072, GFI=0.92, CFI=0.90, AGFI=0.87). The resulting fit indices show that the data can be described with a single factor solution ( Figure 1).
The analysis trying to confirm a single-factor structure of the ICES Father form showed that the

Criterion Dependent Validity
To evaluate the criterion-dependent validity of the ICES, correlational relations between the instrument's Mother and Father forms and s-EMBU as well as RSES were assessed. As it is konwn, the s-EMBU used in this study, like ICES, is an instrument which participants respond to all items seperately for their mother and father. It includes three subscales (overprotection, emotional warmth, and rejection). Table 1 shows the total scores and standard deviations obtained from the instruments.
Evaluating the strength of the correlation coefficient (r) it can be said that, a value of r<0.2 indicates a very weak or no correlation, a value between 0.2 and 0.4 a weak correlation, a value between 0.4 and 0.6 an intermediate level, a value between 0.6 and 0.8 high level and >0.8 very high correlation (35).
Between the ICES Mother form and the s-EMBU subscale perceived Emotional Warmth from the mother, a negative correlation of intermediate level of significance is shown (r=-0.58, p<0.01). There is a significant and positive correlation at a low level (r= 0.34, p<0.01 ) between the ICES Mother form and the mother form of perceived Overprotection subscale of s-EMBU, and also a positive significant correlation at low level (r=0.31, p<0.01) for the rejection subscale too. There is a statistically significant posisitive correlation at low level between ICES mother form and RSES (r=0.34, p<0.01). (On the latter scale, low points indicate low selfesteem.) Between the ICES Father form and the s-EMBU subscale perceived Emotional Warmth from the father, a negative correlation of high level of significance was found (r=-0.67, p<0.01), while for the s-EMBU subdimension Overprotection a significant positive correlation at an intermediate level (r=0.44, p<0.01) and for the Rejection subscale a positive significant correlation at a high level (r=0.71, p<0.01) were found. There is a significant positive correlation at a low level   Table 2.

Internal Consistency
The reliability of the scale is confirmed using Cronbach's alpha internal consistency coefficient, splithalf reliability, and item-test correlation. Cronbach's alpha values vary between 0 and 1, and reliability increases when approaching 1. Cronbach's alpha value above 0.70 is accepted as sufficient for the reliability of the scale (36). Chronbach's alpha internal consistency coefficient for the mother form of the scale was found as 0.84. Correlations between the individual items of the scale ranged between 0.36 and 0.63. The split-half reliability coefficient for the Mother form was found to be 0.75, while for the Father form, the internal consistency coefficient worked out as 0.87. The correlations between the individual items of the Father form are ranging from 0.44 to 0.70. In addition, the split-half reliability coefficient for the Father form was found as 0.81.

DISCUSSION
This study was carried out in order to adapt the ICES to Turkish culture. Although the concept of invalidating environment is part of Linehan's (6) model of dialectical-behavioral therapy, the scale was developed based on the work by Mountford et al. (1). Known under the acronym ICES, the instrument aims to evaluate childhood experiences of invalidation by both mother and father and also tries to establish how an individual in general perceives his or her childhood environment.
The original study examined psychometric properties of the scale as well as its relation with eating disorders. While the study mentioned above and another research done with eating disorder patients found satisfactory Cronbach's alpha values for the clinical groups (1,20), internal consistency coefficients for non-clinical groups were much lower (1,19). By contrast, the present study was carried out in a normal group and internal consistency coefficients for both parents are found to be in a range of high reliability.
Two different tools were used in order to establish the criterion-dependent validity: The short version of the "Egna Minnen av Barndoms Uppfostran: My memories of upbringing" form for children (s-EMBU) and the RSES. In a study examining the psychometric properties of the Invalidating Environment Scale, it was also seen that for criterion-dependent validity parents' attitudes were measured similarly (11). Beginning from the childhood, the seeds of self-esteem starts to develop (41), and ideal parenting helps children to develop an integrated self and a healthy self-esteem (42). A review from the related literature found that parenting styles are one of the main factors affecting self-esteem and behavior in the process of development during childhood (43,44). Thus, selfesteem and perceived parenting styles were used to assess criterion-dependent validity. The self-esteem of the individuals, who experienced less validation during childhood, is being negatively affected" hypothesis was also tested via the analyses and it was found that there is a correlation in the expected direction between these two variables. When the relations between perceived over protection, warmth and rejection from both parents and perception of invalidation, for both parents a negative correlation between invalidation and warmth and positive correlations between invalidation and overprotection as well as rejection have been found. In a study by Huxley and Bizumic (5), parenting styles were assessed integrally rather than individually and a single measurement was taken. Warmth dimension was renamed as coldness after reversyl coding items scored under this dimension. When the results of the current study are compared with the results of the study mentioned above, similar findings was found that, overprotection, rejection and coldness of the parents were significantly positvely correlated with the perception of invalidation. In another study, Robertson et al. (15) evaluated perceived parenting styles with a different instrument consisting of the subdimensions "care" and "overprotection". The researchers found perceived invalidation from both mother and father to be positively correlated with overprotection and negatively with care. The findings of this study are also in line with the findings of the current study. Findings of other studies, that examine the relation between an individual's perception of parenting styles and self-esteem, are also in line with the findings of the current study. Parker et al. (45,46) carried out a series of studies examining the effect of two parenting styles (warmth and overprotection) on mental health, reporting that self-esteem is significantly correlated with perceived warmth and overprotection from the mother as well as with perceived overprotection from the father. Similarly, studies by Lambron et al. (47) examining relation between parenting styles and self-esteem reported that the most beneficial parenting style for the development of self-esteem was an authoritarian one that included warmth and high expectations. The current study also found a significant correlation between self-esteem and perceived warmth and overprotection from the mother as well as perceived overprotection from the father. As in every study, there are certain limitations of the current study. The first limitation is the usage of the scale in a limited sample of university students. This is a significant limitation for the generalizability of the findings. Another problem arises from the nature of the instrument using a retrospective data collection method. Answers given to retrospective data collection instruments may of course be influenced by the person's current emotional state, relationships, and mental signs. On the other hand, the scale which is adapted in the current study is expected to beneficial in filling the gap regarding the constantly needed data collection tools in the related lietrature. This scale, which is previously has been mostly used to examine borderline and narcisisstic personality doserders and eating disorders, is expected to be helpfull in examining not only the topics mentioned above but also its relation with other kinds of psychopathologies in Turkey.
In sum, it can be said that the ICES Turkish form, evaluating retrospectively invalidation experiences from both parents, is a valid and reliable tool that can be used in studies dealing with childhood, parentchild relationship, and family-centered research.