{'fraction_non_alphanumeric': 0.03223322896884463, 'fraction_numerical': 0.030231483857722116, 'mean_word_length': 4.943868212324588, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Realistic Mathematics Education in Indonesia and Recommendations for Future Implementation: A Meta-Analysis Study

Received : 07-02-2020 Revised : 22-03-2020 Accepted : 25-03-2020 Online : 02-04-2020 This meta-analysis study aims to investigate the effectiveness of applying realistic mathematics education (RME) in Indonesia to students' mathematical abilities. This study analyzes 95 effect sizes from 72 studies that have been published in national and international journals or proceedings from 2010 to 2019. Comprehensive Meta-Analysis (CMA) software is used to aid analysis. As a result of the study, the overall effect size was 1.104, with a standard error of 0.065 according to the random-effects model. These results indicate that the average person who is ranked 13th in the experimental group is equivalent to those who are ranked 4th in the control group. This research was carried out by considering four characteristics, resulting in significant differences in terms of sample size and duration of treatment. Thus the application of RME in Indonesia is very effective in improving students' mathematical abilities by considering the sample size and duration of treatment. Keyword:

Various studies have found the application of RME in Indonesia affects the mathematical abilities of students (Hasratuddin, 2010;Putri, 2011;Artawa, Wyn, & Suwatra, 2012;Palinussa, 2013;Rohmah, Caswita, & Widyastuti, 2014;Rohman & Sugiman, 2015;Yunisha, Prahmana, & Sukmawati, 2016;Nursiddik, Noto, & Hartono, 2017;Dwi Yanti, Wahyu Widada, & Zamzaili, 2018;Husniyah, Sulistiani, & Mustafida, 2019). However, to date in the literature, no comprehensive evaluation has been carried out on the effect of RME on students' mathematical abilities. In addition, the researchers have not investigated further the effects of study characteristics such as study years, sample size and education levels, and the duration of the experiment as moderator variables, which might also explain the effect of RME on students' mathematical abilities. On the other hand, the government and related parties need a comprehensive conclusion about the effectiveness of the RME, along with the characteristics that influence its implementation in the future.
Investigating the effect of RME on mathematical ability in terms of the characteristics of the study cannot be done with primary studies. Meta-analysis is seen as an objective method of literature review because it uses effect sizes. This procedure ignores subjective interpretations of various research reviews on the same topic or method (Borenstein & Hedges, 2009;Cohen, 1988). Meta-analysis is to collect the results of studies consistently and precisely (Hedges & Olkin, 1985;Cohen, 1988). Meta-analysis is a quantitative technique that uses specific steps (for example, effect sizes) to show the strength of the variable relationships for the studies included in the analysis ( Cleophas & Zwinderman, 2017;Schwarzer, Carpenter, & Rücker, 2015;Shelby & Vaske, 2008).
Several Meta-analysis studies conducted by (Asror, 2016, Prasetiyo, Yusmin, & Hartoyo, 2014Shelby & Vaske, 2008;Turgut & Turgut, 2018) are analyzing the effectiveness of the effects of learning interventions such as the effect of Problem Based Learning and cooperative models and the use of media on students' mathematical thinking abilities. There is no specific meta-analysis about RME. As a result, an in-depth and comprehensive picture of how the RME effect is seen from various study characteristics such as sample size, the year when the study was conducted, school level, etc. have not been examined. This explanation shows that a comprehensive meta-analysis of the effectiveness of RME on students' mathematical abilities is needed to evaluate its application and see the overall trends clearly. This underlies researchers to investigate the effect of the application of RME on students' mathematical abilities with the help of meta-analysis methods. In this context, this study examines the following questions: 1. Does the application of RME produce a greater effect size on students' mathematical abilities than conventional approaches? 2. Does the effect size of students' mathematical abilities from applying RME between different study groups reviewed from the study year? 3. Does the effect size of students' mathematical abilities from applying RME between different study groups in terms of education level? 4. Does the effect size of students' mathematical abilities from applying RME between different study groups in terms of sample size? 5. Does the effect size of students' mathematical abilities from applying RME between different study groups in terms of the duration of the experiment?

B. METHODS 1. Research Design
This study aims to statistically evaluate the findings of primary studies examining the effect of the application of RME on the mathematical abilities of students in Indonesia, using a meta-analysis method. Meta-analysis provides an overall evaluation with statistical analysis of quantitative data obtained in independent studies on specific subjects (Cleophas & Zwinderman, 2017;GLASS, 1976;& Schwarzer et al., 2015). Effect size is a simple way to measure differences between two groups, which have many advantages compared to using statistical significance tests alone (Coe, 2002;Ellis, 2010;Thalheimer & Cook, 2002). Meta-analysis is carried out by following the steps; First, the criteria for the study included in the meta-analysis will be presented. Second, the procedure for finding studies and coding of study variables will be explained. Third, statistical techniques to investigate the relationship between study variables and effect sizes (Borenstein, Hedges, & Rothstein, 2007;Pigott, 2012). This stage was also carried out in this study.

Inclusion Criteria
The studies included in this analysis were selected from experimental and quasiexperimental studies comparing the achievement of studies taught with RME and students taught with conventional approaches. Studies included in limited synthesis were conducted in Indonesia in the past decade (2010-2019). The statistics needed for this transformation are the mean, standard deviation, and sample size. In addition, the information needed to investigate the research question is the source of the publication, the year of study, and the duration of the treatment.

Study Search
The study included in the analysis was found using an electronic database, the Education Resource Information Center (ERIC), and a journal published by Springer. The keywords used are "Realistic Mathematics Education." Furthermore, to reach journal articles and national seminar proceedings in Indonesia, we use the Google Scholar search engine. The keyword used is "Pendidikan Matematika Realistik." Table 1 shows information about studies that have been published by various journals. As explained in Table 1, the list of journals that published studies on RME was 31. Search results found 216 studies examining the application of RME in mathematics learning in Indonesia from 2010 to 2019. In accordance with the inclusion criteria, studies that were used in the analysis were 72. However, because some studies tested more than one comparison, there were 95 effect sizes analyzed. In a study that did not include the duration of the treatment, the researcher cross-referenced the original author by email in the journal. Table  2 shows the information about the study.

Reliability Test
The instrument in this study was carried out by coding code sheets. To ensure data is entered without error, the two encoders fill out the encoding form separately and then compare. If there is still data that is not the same, then match it again. The most important criticism that is common in meta-analysis studies is publication bias, which is the tendency of journals to publish only significant studies that lead to meta-analyzes that are too high for actual effect sizes (Borenstein & Hedges, 2009;Park & Hong, 2016). One approach, known as Trim and Fill, graphs the relationship between standard errors and effect sizes, with results referred to as funnel plots (Richard, Light, & Pillemer, 1984, WILLETT, 1993. It said there was no publication bias if the study was distributed symmetrically. Figure 1 presents a funnel chart obtained in the study. As explained in Figure 1, the effect size is spread almost symmetrically in the center of the funnel plot and on the left and right sides of the vertical line. Vertical lines show the size of the combined effect. Because the distribution is not fully symmetrical, Rosenthal's fail-safe N (FSN) statistics are helped to determine the probability of publication bias. From the analysis of data with the help of CMA software, the Rosenthal safe N value is 33411. According to the formula N / (5k + 10) (Nursiddik et al., 2017) that is 33411 / (5 * 95 + 10), the calculation results are 68,889. According to this calculation, it can be identified that the studies included in this analysis are resistant to publication bias. Thus it is stated that the results of the meta-analysis in this study are reliable.

Statistics Analysis
Calculation of average effect size and hypothesis testing using CMA software. The effect size used is Hedge's g. Interpretation of effect sizes, using classification (Thalheimer & Cook, 2002) (Borenstein & Hedges, 2009). This means that the application of RME produces a greater effect size on students' mathematical abilities than conventional approaches. When the effect size is statistically heterogeneous (Qb> χ2.95; p <0.05), the hypothesis on the homogeneity of the effect size is rejected (Demir & Başol, 2014). The random effect model is used when Qb is blocked. Rejecting Qb implies that the effect sizes of the study characteristic groups may not measure the same population parameters (Borenstein & Hedges, 2009). In other words, there is a statistically significant difference in the average effect size for each group of study characteristics (Bayir & Bozkurt, 2018).

C. RESULT AND DISCUSSION
The first objective of this study was to determine the overall effectiveness of learning applying RME to students' mathematical abilities. Based on the overall calculation, the effect sizes from each study are presented in Table 3, which is exported from the following CMA outputs: As depicted in Table 3, the overall range of effect sizes is from 0.05 to 3.40, with a 95% confidence limit. Referring to classification (Thalheimer & Cook, 2002), it can be examined that twenty-five effect sizes have very good levels; fifteen effect sizes have very high levels; twenty-six sizes have a high level; nineteen effect sizes have medium levels; the other six have low levels. Only four effect sizes have no level. Table 4 shows a comparison of meta-analysis results according to the effect model. As depicted in Table 4, it appears that according to the fixed-effect model, the lower limit of the 95% confidence interval is 0.964, and the upper limit is 1.076. The average effect size is calculated as 1,020. This effect size is accepted as a high-level effect. Homogeneity test results revealed that the Q value was 469,515. This value was found to be greater than 117,632 in degrees of freedom 94 in table χ2. Thus, the distribution of effect sizes was found to be heterogeneous. Because the homogeneity test results were rejected, the random-effects model was evaluated. According to the random-effects model, the 95% confidence interval has a lower limit of 0.977 and an upper limit of 1230, and the average effect size is calculated as 1,104. This effect size is accepted as a high-level effect. As a result of the calculation of the z test to determine statistical significance, the z score was found 17,069. This result can be said to be statistically significant at the level of p <0.001. Thus, the application of RME results in a greater measure of the effect of students' mathematical abilities than conventional approaches.
Then 95 effect sizes were examined based on the characteristics of the study, namely the year of study, school level, the sample size of the experiment, source of publication, and the duration of the experiment. Table 5 below is a summary of the results of the analysis. According to the random-effects model, the effect size of the study is 1,104, indicating that learning that applies RME has a strong influence on students' mathematical abilities when compared to conventional learning. The effect size of 1,104 indicates that the average student exposed to RME exceeds the mathematical ability of 79% of students in the conventional class who are initially equivalent. This finding can also be interpreted that students shift from the 50th percentile to the 84th percentile in mathematical abilities when RME is applied. Besides the effect size of 1,104, if you are confirming with the interpretation table (Coe, 2002), it can be interpreted that the average person is ranked 13th in the experimental group, equivalent to those ranked 4th in the control group.
This finding does not differ greatly with research (Prasetiyo et al., 2014;Turgut & Temur, 2017), which found an effect size of 0.900 and 0.840 when they each synthesized 22 and 47 studies comparing conventional and cooperative learning models to students' mathematical abilities. Similar results were reported by (Asror, 2016). In his research found an effect size of 0.94 for the effectiveness of the Problem Based Learning model compared to conventional learning on students' mathematical abilities. Similar results were also shown by (Tumangkeng, Yusmin, & Hartoyo, 2018), who found the effect size of 0.95 when they synthesized 31 studies comparing learning using conventional mathematics with media on students' mathematical abilities. This finding shows a new fact that the application of RME produces a larger effect size than conventional approaches, cooperative approaches, and Problem Based Learning as well as learning using media.
This meta-analysis detected significant differences in effect sizes based on study characteristics. When Table 5 was examined, it appeared that the Z scores for all study characteristics were found to be greater than the Z table at the level of p <0.001. This means that the application of RME is more effective than conventional learning in terms of study characteristics. Characteristics based on the year of the study found that studies conducted from 2010 to 2015 had an effect size of 1,099 (high level) almost the same as the effect size in studies between 2016 and 2019 of 1,025 (very high level). The statistical value of Qb obtained as a result of the homogeneity test was calculated as 0.287. This value is smaller than the value of 3,841 at a 95% confidence interval from the 0.05 significance level. This means that there are no significant differences between the study groups. This finding is consistent with other studies conducted by (Bayir & Bozkurt, 2018). The researchers found the effect sizes of studies conducted from 2005 to 2010 did not differ greatly from the effect sizes of studies conducted from 2011 to 2016.
According to the results of the analysis given in Table 5, the effect size in studies conducted in primary schools (PS) of 1,069 (high level) does not differ greatly from the effect sizes in studies conducted in junior high schools (JHS) of 0.969 (high levels ). Both are not much different from the effect size in studies conducted in high school and vocational schools (SHS & VHS), which is equal to 0.848 (high level). The Qb statistical value obtained as a result of the homogeneity test was calculated as 4,309. Because this value is smaller than the value of 5.99 at the 95% confidence level, the significance level is 0.05, so it can be said that there is no significant difference in the effect sizes between groups according to the school level. This result is different from the findings of research conducted by (Asror, 2016) that the application of problem-based learning models as part of constructivism-based learning models is more effective in junior high than in high school.
Judging from the sample size summary of the results in Table 5 shows the size of the effect on studies conducted with a range of 1-30 students amounting to 1,104 (very high level) greater than the size of the effects on studies conducted with a range of 31 or more students amounting to 0.928 (level high). The statistical value of Qb obtained as a result of the homogeneity test was calculated as 9,649. Because this value is greater than the value of 3,841 at the 95% confidence level, the significance level is 0.05, so it can be said that there is a significant difference in effect sizes between groups according to sample size. This result is very different from the findings (Bayir & Bozkurt, 2018). Based on the sample size, the researchers found that small samples have smaller effect sizes than large samples. This difference in results is another issue that can be further investigated.
Based on publication sources, it was found that the effect size on studies published in national and international journals was 1,004 (high level); the size of the effect on studies published on proceedings was 1,020 (high level). The statistical value of Q obtained as a result of the homogeneity test was calculated as 0.030. Because this value is smaller than the value of 3,841 at the 95% confidence level, the significance level is 0.05, so there is no significant difference in the effect sizes between groups according to published sources.
Judging from the duration of the treatment it was found that the effect size in studies conducted between 0-7 hours was 1,268 (very high level); effect size in studies conducted between 8-14 hours was 0.987 (high level); and the size of the effect on studies conducted between 15 hours or more (moderate level). The statistical value of Q obtained as a result of the homogeneity test is 65,563. Because this value is greater than the 5.99 value at the 95% confidence level, the significance level is 0.05, so there is a significant difference in the effect sizes between groups according to published sources.

D. CONCLUSION AND SUGGESTIONS
This meta-analysis was carried out to investigate the effectiveness of the application of RME to students' mathematical abilities. According to the random-effects model, with a 95% confidence interval, the effect size was calculated at 1,104 with a standard error of 0.065. This shows that the application of RME results in a greater effect on students' mathematical ability than conventional approaches. Judging from the characteristics of the study, it produced a significant difference in terms of sample size and the duration of the experiment. Thus the application of RME in Indonesia is very effective in improving students' mathematical abilities by considering the sample size and the duration of the experiment. This finding also shows that RME can be applied at various levels of education.
Although this analysis shows that the application of RME has a very high effect on students' mathematical abilities, this finding is only based on studies that allow the calculation of effect sizes. There are still many similar studies that have not been analyzed because the statistical information needed is inadequate. This research has not yet reached other characteristics such as research sites, types of mathematical abilities, learning materials,