{'fraction_non_alphanumeric': 0.02854455678344698, 'fraction_numerical': 0.017634610862920293, 'mean_word_length': 4.887596899224806, 'pattern_counts': {'":': 0, '<': 7, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A Comparison of Latent Class Analysis and the Mixture Rasch Model Using 8th Grade Mathematics Data in the Fourth International Mathematics and Science Study (TIMSS-2011)

: This study provides a comparison of the results of latent class analysis (LCA) and mixture Rasch model (MRM) analysis using data from the Trends in International Mathematics and Science Study – 2011 (TIMSS-2011) with a focus on the 8th-grade mathematics section. The research study focuses on the comparison of LCA and MRM to determine if results obtained differ when the assumed psychometric model differs. Also, a log-linear analysis was conducted to understand the interactions between latent classes identified by LCA and MRM. Response data to the three booklets were used to run latent class analysis using Mplus 7.31 (Muthén & Muthén, 2012a) for LCA and WINMIRA (von Davier, 2001a). The findings of this paper do not reveal unequivocally whether a model based on primarily qualitative differences (LCA), that is, different strategies, instructional differences, curriculum etc. or a model including additional factors of quantitative differences within strategies (MRM) should be used with this particular dataset. Both of the tests provided similar results with more or less similar interpretations. Both techniques fit the data similarly, a result found in prior research. Nonetheless, for tests similar to TIMSS exams, item difficulty parameters can be useful for educational researchers giving potential priority to use of MRM.


INTRODUCTION
Latent class analysis (LCA) is a subgroup of structural equation modeling which is used to find categorical groups or subtypes of cases, in the present case based on responses to test items (McCutcheon, 1987). Mixture Rasch models, which combine Rasch models with latent class analysis, have been used to identify latent classes who might use different problem-solving techniques or who use different skills in response to test items. The purpose of this study was to compare of the results of latent class analysis and mixture Rasch model analysis for a major international assessment in mathematics. Latent class analysis and mixture Rasch model analysis are two approaches to identification of latent classes in data. The purpose of the two approaches and likely the outcomes overlap but assumptions about the nature of the data and the information derived from each approach differ. The existence of multiple latent classes in test data speaks to the validity of test scores, particularly with the mixture Rasch model. If multiple latent classes are found in test data, distinct groups of participants exist for whom the construct varies, making cross-country comparisons suspect. In this study, results of two statistical techniques for latent class estimation based on students' responses were compared.

Latent Class Analysis and the Mixture Rasch Model
Since both techniques are used in educational sciences, it is important to summarize their similarities and differences. Rasch models assume that participants who have the same ability have similar item solution techniques, skills, and psychological procedures used for solution (Fischer & Molenaar, 2012). However, studies in cognitive psychology and standardized testing have suggested that participants at the same ability level might use totally different techniques and strategies and take different paths to arrive at a solution (Sigott, 2004;Sternberg, 1985). If so, the test construct may change for different participants depending on the paths they take for solving the items, which is a threat to construct validity. LCA and the MRM are statistical models used to examine this threat. Analysis of examinee responses to test items typically rests on the assumption that item parameters are homogeneous across examinees; that is, the items are assumed to behave in the same way for all examinees. In a conventional Rasch analysis, a single difficulty parameter is estimated for each item, and all item difficulty estimates are located on a single dimension along with a single ability parameter for each examinee. However, when examinees systematically differ in the ways they understand or solve items, this assumption may no longer hold. Differences in item solution processes, for example, can give rise to differences in item position parameters and hence to different latent classes. The fundamental concept underlying LCA is straightforward: some of the parameters of a statistical model differ across unobserved subgroups. These subgroups, which are posited to be nations in this case, are the categories of a categorical latent variable (Vermunt & Magidson, 2004). The mixture Rasch model, on the other hand, is based on the Rasch model (Rasch, 1960), and was introduced by Rost (1990). It is a mixture of a latent trait approach and a latent class approach to model qualitative and quantitative ability differences. The model assesses a set of items as a whole. Therefore, it is the set of item parameters for all items that is tested for differences between latent classes rather than each item parameter being tested individually (Frick, Strobl, & Zeileis, 2015). LCA estimates relationships between indicator variables due to class membership only. Also, it calculates class membership probabilities instead of fixed class memberships. For example, if there are four suspected classes in a data set the probability of a participant being in each class might be as follows: 0.76, 0.14, 0.08, and 0.02. Since LCA does not provide fixed class memberships for each case, another step takes place within the model selection process called "quality of the classification of latent class membership" (Wang & Wang, 2012). A criterion value from Nagin's (2005) study is used to determine the quality (.70 and higher). Finally, LCA requires each latent class to be defined in a meaningful manner so variance within the population can be described. As a result of this, latent class interpretation is a very important step of LCA. However, in the MRM, because each class of participants shows a different pattern of response, there are different parameter estimates for each class. The class-related differences in item parameter estimates (the relative difficulty of items) provides differences in how the construct being examined is understood by that class's respondents. Unlike LCA, the class assignment method the MRM uses is a fixed assignment procedure called modal class. One important point is that LCA's path for class membership divides the sample into different groups. Final class membership probabilities provide percentages rather than fixed class membership. At first, one might emphasize that LCA's procedure can provide statistical optimization. However, while gaining statistical optimization, classification interpretability and usability can be lost. Also, in the case of a follow up study with same participants, 72% of one case cannot be invited to a focus group while 28% of the same case stays in another group (Dallas & Wilse, 2013). The solution the mixture Rasch model provides on this matter is using item difficulty parameters. Since the main product of each class is item difficulty parameters, interpretation of classes is derived from differences in item difficulties. Therefore, there is no need to evaluate the quality of the classification of latent class membership, and to define the latent classes for modeling purposes in the MRM.

METHOD 2.1. Research Goal
This study evaluated and compared the performance of LCA and MRM methods. Both techniques were used in terms of questionnaire validation to see if TIMSS-2011 data yielded different sub-groups within the selected nations.

Sample and Data Collection
Data used in this study were taken from the TIMSS-2011 8th grade mathematics section administered in 2011. Students' responses to the items were used for analyses. There were 26.596 8th grade students from four different nations. The reason to select these nations was mainly their performance shown on the exam and their cultural differences. For country specific descriptive information, see Table 1. The TIMSS-2011 8th grade mathematics test consisted of 217 items which included 118 multiple-choice items in 14 different booklets. Each booklet contained 10-18 items. Six of the mathematics blocks were released. Only Booklets One, Four, and Six were used due to having a larger number of released items in those booklets but only results from booklet six will be discussed. The total number of released items included in these booklets is 40.

Analyzing of Data
Response data to the three booklets were used to run latent class analysis using Mplus 7.31 (Muthén & Muthén, 2012a) for LCA and WINMIRA (von Davier, 2001a). Competing models were selected by using information criterion values which the Bayesian information criterion (BIC) and Akaike's information criterion (AIC) for LCA and Pearson Chi-square value and Cressie-Read statistic (Cressie & Read, 1984) for the MRM. Although the original study contained 3 different booklets results from booklet four will be used due to limitations on word count. Also, it is important to emphasize that both techniques provided similar results for booklets one and six.
The latent class structure of the TIMSS-2011 8th-grade mathematics data was assessed by both analyses. Following that a log-linear analysis was conducted to see if defined latent classes were similar.

Latent Class Analysis
The fit statistics and information criterion indices for the models, which ranged from 1 to 4 latent classes, are shown in Table 2. Based on the p-values of the LMR LR test (p = 0.29) and the BLRT test (p = 0.14), both were statistically nonsignificant at the 4-class model; hence, the test failed to reject the 3-class model in favor of a four or more class model. Also, nondecreasing BIC (21392) of the 4-class model supported evidence for the 3-class model, the non-decreasing AIC (21207) of the 4-class model supported evidence for the 3-class model. Hence, the fit of the 3-class model was decided to be adequate and the selected model for further analysis for Booklet Four.

Classification quality
The final class sizes and percentages for the latent classes are given in Table 3. Table 3 shows that 473 students (27.1%) were assigned to Class 1, 694 students (39.0%) were assigned to Class 2, and 579 students (33.9%) were assigned to Class 3. The average latent class posterior probabilities for the most likely latent class membership are reported in Table 4. The probability for most likely latent class membership for students assigned to the first class was 0.87, while the probability of misclassification was 0.13. Similarly, for students assigned to the second class, the probability of correct class membership was 0.76, while the probability of misclassification was 0.24; for students assigned to the third class, the probability of correct class membership was 0.80, while the probability of misclassification was 0.20. All average latent class probabilities for most likely latent class membership exceeded 0.70. Furthermore, entropy was .69 which show that latent class membership classification quality was adequate enough for the 3-class model.

Definition of latent classes
The differences in the sample population were explored by analysis of the estimated itemresponse probability of endorsing "Correct Response" for each of the 10 items. The three latent classes-highly skilled students, moderately skilled students, and somewhat skilled studentswere labeled by the researcher based on the observed pattern of item response probabilities.
The highly skilled students' class, denoted as Class 1 consisting of 473 students, had the highest item-response probabilities for each of the 10 items. Class 2, which contained 694 students with the second highest item-response probabilities for each of the 10 items, as moderately skilled students; Class 3 was defined as somewhat skilled students, which contained 579 students and had the lowest item-response probabilities for each of the 10 items. The unconditional latent class probabilities and the conditional probabilities for endorsing "Correct Answer" are reported by latent class in Table 5. Conditional probability profiles for endorsing the "Correct answer" for the 3-Class model are shown in Figure 1.

Figure 1. Conditional Probability Profiles of Endorsing "Correct Answer" for 3-Class LCA Model for
Booklet Four (Mplus Version 7.31).

Mixture Rasch Model
The dataset consisted of 10 items with 1746 participants. To determine the appropriate number of classes, one, two, three, and four latent class solutions were fit to the data (see Table 2). Pvalues for Booklet Four of Cressie-Read and Pearson Chi-square were .13 and .15. Since the two-class model had the highest p-value, a two-class solution was selected for Booklet Four.
Class size values for each class presents that class 1 was expected to include about 66% of the sample. Class 2 was expected to include about 34% of the sample. The class sizes indicate that about 66 percent and 34 percent of the sample can be fitted by a mixed Rasch model which was assumed to hold in these classes. According to the Q-index, there was one item (M032662) with a Zq value of 2.37 and p-value of .01 which shows lower discrimination in class one. In such cases, item removal is suggested from the scale only after examining the items content and additional information from the estimated model (von Davier, 2001b). Item category values for this item were acceptable. Out of 1,746 responses 1,251 students answered the item false and 495 students answered correct. Additionally, the item parameter value for class one was also acceptable with a value of .13. After examining the item category values and item fit, it is decided not to remove the item from analysis. All of the other items fit each class well (.05 <p< .95) (see Table 5). Figure 2 shows that the two classes had similar item difficulty parameters for the first six items and different item difficulty parameters for the last four items. These four items were slightly easier for first class then for the second class. The lines display items on which the two classes seem to diverge and later to converge. The majority of items were not markedly different in difficulty across classes. In general, all classes found the items to be relatively easy as logit position was generally negative (see Table  6 for specific values including standard error).  The significant p value (< .001) shows that there was a first order effect. The addition of a second order effect improved the likelihood ratio chi-square by 1894.55. This was also significant. But the addition of a third and a fourth order term did not significantly improve fit (p> .05).  Table 8 shows that there were statistically significant associations between nation and LCA class membership (p< .05), nation and the MRM class membership (p< .05), LCA class membership and MRM class membership (p< .05), and nation and gender (p< .05) for Booklet Four. All other interactions between other variables were not statistically significant (p> .05). Due to the purpose of this paper only the association between LCA and MRM results will be explained.
To further analyze the interactions of LCA class membership, and the MRM class membership variables a custom model was created with the significant two-way associations. a. One degree of freedom is subtracted for each cell with an expected value of zero. The unadjusted df is an upper bound on the true df, while the adjusted df may be an underestimate.
In Table 9, the goodness of fit test showed that the model fit the data adequately (p> .05). Also, a crosstab analysis for Booklet Four was done to see LCA class memberships and the MRM class membership agreement level. Although LCA and MRM analysis provided a different number of classes for Booklet Four, LCA's class one (highly skilled students) overlapped 100 % with MRM class two. LCA class two (moderate skill students) overlapped with both MRM class one (81.3%) and class two (18.7%). LCA class three (somewhat moderate skilled students) overlapped with only MRM class one (see Table 10). Please present the findings/results in this section. This section should give significant results obtained from the study clearly and concisely. Please present the findings/results in this section. This section should give significant results obtained from the study clearly and concisely.

DISCUSSION and CONCLUSION
For item parameters, both of the techniques calculate item logit values and standard errors. For LCA, item parameter estimates are on the logit scale, and therefore, can be somewhat difficult to interpret. The same information is given in a more interpretable scale under the MRM where item parameters are products of item difficulty measure for each class. However standard errors of the parameters have very close results for Booklet 4 (see Table 6) The decision on number of classes differs in the two techniques. BIC and AIC were used to evaluate fit for LCA. On the other hand, since Winmira2001 considered data as being sparse, Cressie-Read and Chi-square values were used for model fit purposes. However, based on BIC values, both techniques provided similar results (see Tables 2). So, it can be concluded that selecting one model over another model did not depend on fit values. Since a qualitative conclusion is important for LCA, model fit is not enough by itself. There are also other combinations of different values such as average estimated posterior probabilities for quality (Nagin, 2005) and entropy value (Clark, 2010). Moreover, latent classes should be defined in an interpretable way as well. For the MRM, the solution is simpler. If there is model fit based on fit indices the next step is simply interpretation of the model. The two analyses had somewhat different solutions for the class weights for all booklets. It can be interpreted that latent class analysis puts the most cases into the middle class for three class solutions and to the second class for two class solution. LCA uses response probabilities in which students have the same probability of giving the correct answer within the same class. As a result of this, students in the same class have no quantitative differences. The only difference created and shown by LCA is between groups which is a product of qualitative differences. In our case, this would be interpreted as item correct response values based on students' background. However, the mixture Rasch model, regardless of number of classes within the solution, sorts classes based on similarity in their response patterns which results in the placement of cases with an order where most student fall in to the first class, then second, then third etc. Since there are differences between item parameters within the same class for the MRM, interpretation changes and relies on two things: one being latent class membership and two being the class specific quantitative person parameter (Büsch, Hagemann, & Bender, 2010). This study provides useful information about two commonly used techniques in educational research. Since the data used in this study are from a real data set, none of the techniques were tested under controlled circumstances such as different levels of amount and type of missing data, presence of outliers, sample size (bigger, smaller), item distributions, score distributions, etc. Monte Carlo simulation studies are recommended to see if the results differ under these different conditions. Further, TIMSS multiple-choice items were dichotomous; use of items with varied responses scales is also recommended, as are studies with item content very different from a mathematics achievement test. For example, studies are recommended that compare LCA and MRM when the construct assessed is a personality variable or attitudinal as well as achievement. The comparison of both techniques is limited to dataset used in this study. Therefore, it is suggested that same study can be done using other type of questionnaires.
As with any statistical approach that uses binary variables, recoding categorical responses into dichotomous responses was one of the limitations of the study since student responses might result in different classification based on the multiple-choice responses. In any latent class model, the issue of reification is of great importance. Also using a real-world dataset limited the radius of effect area of the study since conclusions are limited to the current data. Sampling techniques of TIMSS organizers is also another limitation. One simple example shows that number of students in Turkish and American educational systems are more than the whole population of Singapore and Finland. TIMSS requires each participant country to join with at least 4.500 students. Although this number covers most of the Singaporean and Finnish 8th-grade population, it is still small for systems like the US or Turkey (Rutkowski & Rutkowski, 2016). In this case, generalizability of the results is questionable.

APPENDIX
Appendix A: LCA 2 Class Model Specification for Booklet One (Other Classes Similar) (Mplus Version 7.11).