{'fraction_non_alphanumeric': 0.026472243290191568, 'fraction_numerical': 0.012086218285910014, 'mean_word_length': 5.210910196018842, 'pattern_counts': {'":': 0, '<': 6, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Metacognitive errors in the classroom: The role of variability of past performance on exam prediction accuracy

Students often make incorrect predictions about their exam performance, with the lowest-performing students showing the greatest inaccuracies in their predictions. The reasons why low-performing students make inaccurate predictions are not fully understood. In two studies, we tested the hypothesis that low-performing students erroneously predict their exam performance in part because their past performance varies considerably, yielding unreliable data from which to make their predictions. In contrast, high-performing students tend to have consistently high past performance that they can rely on to make relatively accurate predictions of future test performance. Results showed that across different exams (Study 1) and different courses (Study 2), low-performing students had more variable past performance than high-performing students. Further, results from Study 2 showed that variability in past course performance (but not past exam performance) was associated with poor calibration. Results suggest that variability in past performance may be one factor that contributes to low-performing students’ erroneous performance predictions.

difference between low performers' performance predictions and their actual performance than there is between high performers' performance predictions and their performance.
Calibration errors can be problematic, especially for individuals who overestimate their knowledge and abilities. Overestimations of ability may lead people to engage in challenges that they are not prepared to tackle or to perform tasks, even dangerous ones, that they cannot accomplish successfully. The potential problems of overconfidence are perhaps best highlighted by studies showing that people overestimate their gun safety knowledge (Stark & Sachau, 2016). But, overconfidence can lead to other serious (though less dangerous) problems, especially for students. In educational settings, being able to accurately monitor one's current knowledge allows for an efficient learning process, such that students can use the information from their monitoring to inform their study methods and select poorly-understood topics for further study (Dunlosky et al., 2005;Dunlosky & Ariel, 2011;Dunlosky & Rawson, 2012;Thiede, 1999). This process breaks down for students who have poor calibration. For low-performing students, they err on the side of predicting that they know more than they do. The problem with erring on the side of overconfidence (rather than underconfidence) is that students may decide that additional preparation is unnecessary, leading them to perform poorly on exams and other assignments.
Because of the potential problems with inaccurate metacognitive monitoring, there's been a great deal of research aimed at understanding the source of these metacognitive errors. For example, one theory suggests that metacognitive errors arise in response to poor learning processes especially in the low performers, whereby lack of knowledge of the material also yields lack of awareness (e.g., Kruger & Dunning, 1999). There are many reasons for this lack of knowledge and accompanying lack of awareness, including the possibility that students do not always engage in effective study strategies, such as self-testing, which would yield accurate knowledge (Karpicke & Roediger, 2008). There is also evidence that low-performing students are not particularly confident in their predictions (Miller & Geraci, 2011) and that they rely on other factors when making predictions, including the desire to perform well, which leads them to make high performance predictions (Saenz et al., 2017Serra & DeMarree, 2016). According to the information-motivation theory (Tirso & Geraci, 2020), low performers err in the direction of overconfidence when they lack useful information because they opt to rely on motivational information to aid their predictions of future performance. This theory is consistent with prior research showing that people's predictions about the future are often based on their aspirations rather than their past behaviors (e.g., Helzer & Dunning, 2012).
The current studies were designed to examine another possible contributor to poor calibration: variability in past performance. We hypothesized that low-performing students erroneously predict their exam performance in part because they rely on a history of variable past performance when making predictions. In other words, the lowestperforming students may have obtained some high grades and some low grades on past exams. Reliance on this type of noisy data may present low-performing students with a difficult signal extraction problem whereby they may be unsure which of their prior grades is most indicative of their true level of ability, opting to focus on their higher grades. In contrast, high-performing students may have more consistently high past performance than low-performing students. Thus, high-performing students would have more consistent and reliable information upon which to base predictions of future test performance.
There is some research that is consistent with this hypothesis. We know that, in general, the association between any two constructs can be attenuated by increased measurement error in either or both constructs (e.g., Crocker & Algina, 1986;Worthen et al., 1999). That is, when observed variables (e.g., attention and academic performance) have random measurement errors, their relationship to each other is weaker than the relationship between the true scores of these variables. So, students with less consistent academic records can be thought of as having more measurement error than those with consistent records, whereby the reduced consistency in prior performance would attenuate any association between their true ability and their metacognitive judgments. Also, if we conceptualize prior performance as a cue that students use to varying degrees when making metacognitive judgments (e.g., Koriat, 1997), then that cue will be less reliable (and thus have less utility) for students with noisier histories.
We tested the hypotheses that low-performing students have more variable past performance than high-performing students and that this increased variability is associated with worse calibration. We focused on global predictions using a measure of absolute accuracy (how close a student's overall exam score prediction is to the exam score itself), rather than on relative accuracy of individual test items, for example, which is often measured using gamma to correlate judgments with target performance (see Dunlosky & Thiede, 2013 for review). We chose to focus on global predictions of exam performance because we think these are common assessments that students naturally make, the accuracy of which can have important consequences for their study behaviors and assessments of the course or their major. In Study 1, we examined performance data from several undergraduate psychology courses to determine whether low-performing students' exam grades were more variable than high-performing students' exam grades. In Study 2, we examined whether variability in academic performance across students' classroom tests and across their undergraduate careers predicted calibration on tests within a single course.

Study 1
To examine whether low-performing students showed greater variability in test performance than high-performing students, we examined variability in exam performance using data from several introductory and intermediate-level psychology courses that each included multiple exams.

Participants
Six hundred and forty-nine undergraduate students' deidentified class exam data from a large southwestern university were analyzed. The data were originally collected across multiple semesters from Introduction to Psychology and Cognitive Psychology courses, all taught by the same instructor. Because the policy for these classes was to allow students to drop their lowest exam score, many students opted not to take one of the four exams. After excluding these participants' data, the final dataset consisted of 368 students with data for four exams. We chose to examine data from students with scores on all four exams (rather than just three exams) to obtain a more reliable measure of variability.

Materials and procedure
Data used in the current analyses were gathered from Introduction to Psychology and Cognitive Psychology courses offered across eight semesters (6 classes from fall semesters and 2 classes from spring semesters). In each course, students took four exams as part of the normal course requirements. The format of the exams was multiple-choice, and each exam consisted of 33 questions. The final exam in all courses was cumulative.
Students' exam scores were compiled and de-identified. To examine whether variability in performance was greater for low-performing students compared to high-performing students, we sorted students into low-and high-performing groups based on their performance on a cumulative exam score (the total of all exam scores). This score was then used to divide the students into four quartiles, such that high performers' average grades were above the 75 th percentile and low performers' average grades were in the 25 th percentile and below. This process resulted in 93 low-performing students and 93 high-performing students.

Results
Variability was assessed using the standard deviation of each student's exam scores (see Table 1 for students' average exam scores by performance quartile). Based on examination of standard deviations, the results were consistent with our hypothesis that low performers exhibited more variability in their exam performance compared to high performers, t(138.28) = 13.20, d = 1.935, p < 0.001, 95% CI [6.8,9.19] (Levene's test for homogeneity of variance was violated, therefore, the corrected degrees of freedom 1 are reported). One might worry that sorting students into high and low performers may ensure that variability will be lower in high performers due to a ceiling effect. Thus, it may be helpful to examine variability in performance for students performing in the upper middle-range, as they would be expected to be less variable than the low performers, but this pattern of reduced variability would not be attributable to a ceiling effect. Therefore, we also compared mean variability for high performers (above the 75 th percentile), upper middle performers (between the 50 th and 75 th percentiles), and low performers (below the 25 th percentile); see Table 2. Results showed that the high and upper middle performers had significantly lower mean variability than low performers (all t's > 5.16, p's < 0.05), which supports our hypothesis that performance and variability are negatively related.
Though our goal was to use courses with the best opportunity for observing performance variability-those with four exams-we note that the results are consistent using the larger sample, which included courses with just three exams in addition to those with four exams: Low performers had more variability in their exam performance compared to high performers (average variability of low performers = 19.06; average variability of high performers = 10.91; t(322) = 77.991, d = 8.413, p < 0.001, 95% CI [9.097, 7.727]).

Study 2
Results from Study 1 supported our hypothesis that, relative to high-performing students, low-performing students exhibit greater variability in their exam grades across the semester. The goal of Study 2 was to determine whether variability in past performance is associated with prediction accuracy. We examined two types of variability: variability in past exam performance within a single course, similar to Study 1, and variability in past academic performance as measured by past course grades, across an entire university academic record. Students were recruited from six different classes over the course of a single semester. Just before taking each of their regular exams, students were asked to indicate how well they thought they would perform on the exam. These grade predictions were compared to actual performance on the tests. Variability of past performance was determined both by students' performance across the exams in the class and by students' past course performance across semesters based on their overall college record. We predicted that greater variability in students' academic records would be associated with poorer metacognition.

Participants
Data were collected from a new set of six different psychology classes at a large public southwestern university. These classes included Psychology of Women (Class 1), Introduction to Statistics (Class 2 and Class 3), Research Methods (Class 4), Psychology of Women of Color (Class 5), and Cognitive Neuroscience (Class 6). All six courses used a traditional in-person lecture-based format (all data collection occurred before the COVID-19 pandemic), and students in all six classes were offered extra credit in their course for participation in the study. Out of a total of 482 students, 440 consented to participate in the study. Of these 440 students, 436 (99.1%) consented to release their exam grades to the study authors for inclusion in the analyses, and 395 (89.8%) consented to provide an electronic copy of their unofficial university transcript so that variability of past performance could be determined. Of these 395 students, only 76 (19.2%) provided a copy of their unofficial transcripts. Thus, Study 2 analyses measuring variability in performance across students' academic careers were based on data provided by these 76 students. In contrast, all analyses that measured variability in performance across exams within a single course were based on data from 404 students-the number of students who had both exam and prediction data. Demographic data were not collected to keep the questionnaires administered to students prior to testing as short as possible.

Materials and procedure
All materials and procedures were compliant with the Family Educational Rights and Privacy Act (FERPA) and approved by the IRB. The researchers visited the six classes to recruit participants. Students were told that, should they choose to participate in the study, they would be asked to complete a brief questionnaire immediately prior to each of their course exams in exchange for extra credit in their course and that there would be no penalties if they ceased participation at any time. Students were also told that the data they provided by participating in the study would be kept by the research team instead of their instructor. Thus, their instructor would not know if they participated in the study. Next, consent forms were distributed, and students had the opportunity to opt in to release their exam grades and/or transcripts to the authors for inclusion in analyses; if students opted to release their transcripts, they were asked to provide an email address so that they could be contacted later to share a copy of their unofficial transcript. For all courses, this recruitment process took place within students' classrooms before one of their regularly scheduled lectures. After the initial recruitment session, research assistants visited each class on exam days to remind students of the study and administer the test prediction questionnaire. 2 This questionnaire asked students to predict the numerical grade they would receive on the exam they were about to take. This questionnaire also asked students to predict what they thought the average, highest, and lowest grades on the exam would be, but these data were not used. All predictions were made on a 0-100% scale.
Classes varied in the number of exams, from three to five exams, and the dates on which their exams took place. Out of the six classes in Study 2, two classes (Class 1 and Class 5) had five exams, two classes had four exams (Class 2 and Class 3), one class had four exams with the fourth one being optional (Class 6), and the remaining class had three exams (Class 4). None of the exams were cumulative. See Table 3 for descriptive data.
Once the semester was over and final grades were posted, students who consented to be contacted for a copy of their transcript were sent an email that reminded them of the study and asked them to complete a 2-min Qualtrics survey that allowed them to securely upload a copy of their transcript. The university at which Study 2 took place allows students to download free electronic copies of their unofficial transcript. Before uploading their transcripts, the students were informed that all identifying information from their transcripts would be redacted and assigned a code known only to the third author. Students were then given stepby-step instructions for downloading a copy of their transcript and uploading the file to the survey, after which the survey ended. Six reminder emails were sent to participants over the course of three months to boost response rates. Out of the 395 students that opted to provide a copy of their transcript, 365 students provided useable email addresses. Ultimately, 91 surveys were started, and 76 transcripts were successfully uploaded.

Analytical plan
Variability in exam performance as measured within a single course The analyses were based on data from 404 students-the number of participating students who had both exam and prediction data available-with differences in degrees of freedom being due to missing data. We began by testing whether low-and high-performing students differed in variability of prior exam grades before testing whether this variability moderated the relationship between grade predictions and actual grades. Low-and high-performing students were defined as students whose average grade fell within the first and third tertiles of the performance distribution. In Study 2 we sorted students into high and low performers based on a tertile split instead of a quartile split used in Study 1 because data were available from fewer participants since only 76 students provided transcripts. Similar to Study 1, variability of past performance was determined by calculating the standard deviation of students' exam grades leading up to their final exam; thus, variability was based on performance from 2-3 prior exams. t-tests were first conducted to compare variability in past exam performance between low-, middle-, and high-performing students.
Next, multilevel modeling 3 was used to investigate how variability in performance interacted with numerical grade predictions to predict actual exam grades. Multilevel modeling was used because the data were characterized by a multilevel structure in which students were nested within different classes-a 2-level structure. The outcome variable was students' actual exam grades, which were on the scale of 0-100. Final exam prediction, prior exam variability, and prior exam performance were level 1 predictors, and course average exam grade (grand mean centered) was a level 2 predictor. Final exam prediction was on the scale of 0-100 and centered around their respective class means so that zero represents the average prediction for a given final exam. Thus, positive values represented grade predictions that were higher than the average predictions, whereas negative values represented grade predictions that were lower than the average prediction. Prior exam variability was transformed using a tertile split (0 = lowest variability, 1 = moderate variability, 2 = highest variability). Prior exam average was centered around the class means so that zero represents the average prior exam performance within a given class. Course average exam grade was conceptualized as a measure of course difficulty and calculated as the average of all exam grades across all students for each course. 4 Adding course difficulty to the model allowed us to determine and account for whether differing exam grades across courses influenced the relationship between variability and prediction accuracy, as it was conceivable that differences in the relative difficulty of the classes used in this study may have affected students' grade prediction accuracy.
The model-building process was an iterative one, beginning with the empty, null model (Model 1.0) to serve as a baseline and calculating the intraclass correlation coefficients (ICC). The ICC indicates the percentage of variance in final exam grades accounted for by the clustering variables themselves. We then added additional predictors with each subsequent model and compared each model's overall fit to the fit of the preceding model using a likelihood-ratio test. Random intercept was included in all multilevel models to allow for the variation in final exam grades across classes. All multilevel models were run in R version 3.6.3 (R Core Team, 2020) using RStudio version 1.2.5042 (RStudio Team, 2020), using the packages lme4 (Bates et al., 2015), lmerTest (Kuznetsova et al., 2017), merTools (Knowles & Frederick, 2019), jtools (Long, 2020), reghelper (Hughes, 2020), and ggplot2 (Wickham, 2016). All models also used restricted maximum likelihood estimation and p < 0.05 as the threshold for statistical significance. All p values were calculated using Satterthwaite's formula for degrees of freedom.

Variability in performance as measured across one's college career
We also tested whether variability in performance across one's academic career was associated with poor prediction accuracy. We started by testing whether low-, middle-, and high-performing students differed in the variability present in their GPAs before testing whether this variability moderated the relationship between grade predictions and actual grades. We used a multilevel modeling approach to address this question, but this time we used the variability in students' grades across their college careers (GPA variability) as the measure of variability in performance instead of variability in their exam scores. We investigated whether GPA variability interacted with the relationship between grade predictions and actual performance across all exams. The models also accounted for the effects of course difficulty and the relationship between GPA and GPA variability (GPA variability decreases as GPA approaches ceiling). The analyses that follow were based on data from the 76 students for which transcript data were available.
Because we were interested in predictions across all exams (i.e., a longitudinal design), the structure of the models used differed from the 2-level design used in Model 1. Instead, the analyses that follow all made use of a 3-level design in which exams were nested within students, and students were nested within classes. One noteworthy advantage of using multilevel modeling with a longitudinal design was that it allowed us to make use of all available observations. Therefore, the following models were based on 246 observations (examprediction data points) from a sample of 76 students.
The modeling process began with the empty model (Model 2.0), with additional predictors being added with each subsequent model. The grades students received on each exam served as the dependent variable. Grade predictions were group mean centered based on the average prediction of all of the participants within each class for each exam so that a prediction of zero represented the average prediction on that exam. GPA variability was calculated as the standard deviation of each student's GPA based on their transcript and transformed using a tertile split (0 = low variability, 1 = moderate variability, 2 = high variability). Course difficulty was the average exam grade for a given course as in Model 1, but overall GPA was substituted for prior exam performance to account for the fact that variability in GPA will decrease as GPA approaches ceiling. Grade predictions were entered at level 1, overall GPA and GPA variability were entered at level 2, and course difficulty was entered at level 3.

Variability in exam performance as measured within a single course A pair of
Welch's t-tests indicated that low-performing students (N = 137, M = 7.97, SD = 5.30) had numerically, but not significantly, more variable past performance than middle-(N = 132, M = 6.99, SD = 5.32, t(266.58) = 1.51, p = 0.133, d s = 0.18) and high-performing students (N = 135, M = 6.88, SD = 5.66, t(268.31) = 1.63, p = 0.103, d s = 0.20; see Tables 3 and 4 for descriptive statistics). Full results of multilevel models can be found in Table 5 Interestingly, this interaction was notably closer to the p < 0.05 cutoff prior to adding prior exam performance to the model (Model 1.1b; ps = 0.053). This finding suggests that, although it might appear that prediction resolution decreased as grade variability increased, this potential relationship is confounded by students' prior grades because variability decreases as grades approach ceiling.

Discussion
In summary, we examined the relationship between variability in prior performance and grade prediction accuracy in two different ways using multilevel modeling-across tests within a single course and across all our students' prior coursework. Within a single course, we found that (1) higher performance on earlier exams was associated with higher performance on the final exam, (2) higher course averages (in other words, easier courses) were associated with higher final exam grades, and (3) higher grade predictions on the final exam were associated with higher grades on the final exam (indicating that students had some degree of accuracy in their predictions, on average). Most importantly however, the hypothesized interaction between predictions and variability (within the span of a single course) was absent. However, this interaction was notably closer to the p < 0.05 cutoff prior to adding average prior exam performance (not variability) to the model (Model 1.1b; ps = 0.053). This finding might suggest that, although it might appear that prediction resolution decreased as grade variability increased, this potential relationship is confounded by students' prior grades and this ceiling effect. When variability in performance was measured across courses, greater variability in the academic record was associated with reduced grade prediction accuracy. The main effects we observed meant that (1) higher grade predictions were associated with higher actual grades on the exams, (2) higher GPAs were associated with higher exam grades, and (3) higher course averages (easier courses) were associated with higher exam grades. Relevant to our predictions, there was a significant interaction between grade predictions and GPA variability. This interaction showed that the association between predictions and performance was stronger for the lowest variability group than for the highest variability group (see Fig. 1). This finding is consistent with our hypothesis that variability in past performance, which is primarily a problem for low-performing students, leads to worse metacognitive accuracy, at least when variability is defined across one's academic career. It is important to note that in the final model, the interaction between GPA variability and grade predictions remained significant even after accounting for students' overall GPAs despite the clear relationship between GPA and GPA variability (as GPA approaches ceiling, GPA variability must approach zero). This finding suggests that GPA variability provides incremental validity in predicting the relationship between students' grade predictions and actual grades that is not captured by a student's average level of ability; this finding also highlights the need to disentangle variability in performance from overall performance.

General discussion
Two studies were designed to test the hypotheses that low-performing students have greater variability in their past performance compared to high-performing students and that increased variability is associated with poor calibration. These hypotheses were tested by examining the variability in students' past performance across exams within a single course (Studies 1 and 2) and across courses within an academic career (Study 2). Results showed that across different exams (Study 1) and different courses (Study 2), low-performing students had more variable past performance than high-performing students. Increased variability in performance across one's academic career (but not across exams within a course) was associated with increased errors on future exam and class predictions in Study 2. Thus, results showed that low performers have more variable past performance than high performers, and that variability across one's academic career is associated with increased prediction errors.
The results showing that variability in performance across tests is not associated with increased calibration errors might have arisen because variability was somewhat limited in Study 2. For example, some courses had only three exams, thus limiting our estimates of variability to just the two exams prior to the one for which students made predictions. Thus, it is possible that using more tests, one would be able to observe greater variability in exam performance, which would be associated with worse calibration. Supporting this hypothesis, the average variability in test performance within a single course was higher in Study 1, using four tests, than it was in Study 2 using between two and five tests. In Study 1, the average variability was 13.87 for low performers and 5.88 for high performers, whereas in Study 2, it was 7.97 for low performers and 6.88 for high performers.
Of course, it is possible that variability within a single course is simply not associated with students' prediction accuracy on a later test. Other studies (Foster et al., 2017) have examined students' use of past performance in making predictions and found no relationship between students' prior exam performance and their predictions. However, Foster et al. (2017) examined the role of the prior test performance from a single test and not variability of all prior tests on subsequent calibration. But, the fact that students in their study did not rely on their performance on the most recent prior test when making predictions suggests that students may not pay a lot of attention to their prior test performance within a single class when making future test performance predictions.
Results from Study 2 showed that increased variability across one's academic career is associated with poor calibration. Low-performing students possessed far more varied academic records than high-performing students. High variability in one's academic record was clearly associated with worse calibration, showing that this led to near zero ability to predict one's performance (see Fig. 1). Given that low-performing students possessed more varied prior performance than high-performing students, and that variability in prior course performance was related to calibration, variability in past performance might provide one explanation for why low-performing students exhibit poor calibration.
We offer one explanation for why low-performing students may have poor calibration in general. But, we note that the typical pattern is not simply one of poor calibration, but also one of over predicting performance. Thus, one may ask why low-performing students are inaccurate in the direction of overconfidence, rather than sometimes under predicting and sometimes over predicting their performance. The design of Study 2 does not allow us to determine the reason for the typical direction of the calibration errors. However, we speculate that, with less reliable information available and a tendency to attribute poor performance to external rather than internal causes (Hacker et al., 2000), low-performing students may turn to other sources of information when making their predictions. For example, research suggests that low performers report relying on motivational factors, such as the grades they wish to earn, more than high performers do, and this reliance on motivational factors predicts poor calibration in these students (see Serra & DeMarre, 2016;Saenz et al., 2017). Because low-performing students' desired grades typically exceed their actual performance (Saenz et al., 2017), this would result in overconfidence, especially when combined with a lack of reliable information. Thus, consistent with the informationmotivation theory (Tirso & Geraci, 2020), we suggest that in the face of impoverished information about prior performance, low performers may rely on other factors that lead them to err in the direction of overconfidence.
It is also possible that, with increased variability in past performance, low performers may choose which past performance they believe or hope is most representative of their abilities. That is, they may select to remember their past successes when predicting the future, choosing to focus on past "wins" rather than past "losses", so to speak. This variability in past performance could allow people to focus more on their potential rather than their past, given that their past performance doesn't yield a clear pattern of performance. Research shows that people tend to think of themselves in terms of who they want to be, instead of who they have been in the past (e.g., Williams & Gilovich, 2008), and their predictions about the future tend to be based on their aspirations (Helzer & Dunning, 2012). So it may be that, if there is a lot of variability in past performance, this pattern allows people to believe in future possibilities. Regardless of the potential mechanism for overconfidence, the current results show that low performers have more variable past performance than high performers and that the increased variability in one's past academic record is associated with increased calibration errors.
As with all studies, this study has some limitations. For example, our ability to examine the relationship between variability of past exam performance and predictions was limited by the number of exams in each course in Study 2. Future studies should examine the role of variable past performance in situations where there are more tests to gain a better estimate of variability to investigate the potential relationship between past performance variability and prediction accuracy. We only examined global performance predictions ("What is your predicted grade"), but future studies might examine whether past performance variability influences local performance judgments ("What is the likelihood that this answer is correct?"). On the one hand, there is likely additional variability inherent in local judgments, as even the top-performing students will miss some items and experience variable item performance. Therefore, one might predict a smaller variability difference between high and low performers for local judgments compared to global judgments. Alternatively, one might predict that low performers would miss some items and get others correct, whereas high performers might get these items mostly correct, such that differences in variability for these groups of students would persist. One might also assess the relationship between performance variability and postdictions (performance predictions that are taken after the test is completed). One hypothesis is that postdictions might be relatively unaffected by past performance variability because people would have recent diagnostic test experience to inform their judgments. Also, the current studies examined the potential role of past performance variability in prediction accuracy in a sample of undergraduate students, so we don't know how the results would generalize to other student or community samples. Further, we did not measure the other factors that may influence prediction accuracy (e.g., course type, perceived course difficulty, type of prior courses, college major, test type: cumulative or not, etc.). For example, it would be interesting to examine whether performance across courses varies less within a students' major compared to outside the major, and whether this influences the accuracy of performance predictions. Also, future studies might examine which courses students are considering when they are making assessments of future course performance. Future studies might examine the contribution of these and other factors to a potential association between performance variability and performance predictions. Finally, the sample size (N = 76) was relatively small in Study 2 due to the limited number of students that provided copies of their unofficial transcripts. Future research is needed to replicate these findings using a larger sample.
There are several practical implications of these results. For example, the results have implications for the type of feedback one might provide to help low performers, in particular, improve their calibration. One suggestion would be to provide students with the full range of their past performance, drawing their attention to the low and the high grades. The results from Study 2 showed that variability in past academic performance predicts calibration, which may mean that students are selectively recalling or choosing which past grades to consider when predicting their future performance. Therefore, it may be important to provide students with the full range of their past performance to impede the potential for selective recall. Even if students do consider the full range of their past performance, they may determine that they will perform well on an upcoming exam because they have performed well before. After all, most students have performed well in at least one course. Therefore, it may be important to draw student's attention to, not simply the full range of past performance, but also the frequency or average of their past performance. It may also be helpful to warn students about the potential pitfalls of variable past performance.
In addition to having implications for the type of feedback that might be effective for helping students improve their calibration, the current results offer another factor for instructors and institutional researchers to consider when determining which students might benefit from extra support, and what type of support they might need. The current results suggest that variability of past performance and not just overall GPA may signal a student in need of certain forms of academic support. A student with a 2.5 GPA with consistent performance may be quite different from a student with the same GPA but variable performance. Both students could likely benefit from extra support in the form of tutoring, learning about effective study strategies, and time management, but the student with consistently poor performance may have different challenges than the student with variable performance, and so the interventions for the two students might be different. Of course, future research is needed to directly test these hypotheses and explore the efficacy of potential interventions, but for now the current results suggest that variability in past performance is an important factor to target for future interventions.
In sum, we found that low-performing students had more variable past performance within a single course (Study 1) and across their academic careers (Study 2) than highperforming students. We also found that variability in one's academic career (but not in a single course) was related to calibration on an upcoming exam. These results suggest that variability in past performance may be an important factor that contributes to students' metacognitive knowledge, particularly among low-performing students.
Author contributions The first author conceived of the studies, contributed to the design of the studies, facilitated data collection for Study 1, and contributed to the writing of the manuscript. The second author contributed to study design and data analysis for Study 1, and to the writing of the manuscript. The third author contributed to the study design, data collection, and data analysis for Study 2, and to the writing of the manuscript. The fourth author contributed to the data scoring and data analysis of Study 2, and to manuscript preparation. The last author contributed to the data analysis, data interpretation, and writing of the results for both studies.

Conflict of interest
The authors report no relevant financial or non-financial interests to disclose. The authors declare that they have no conflict of interest. This study was approved by the Texas A&M University Institutional Review Board (#2019-0783D).
Informed consent Informed consent was obtained by all participants in the studies.