{'fraction_non_alphanumeric': 0.029646902065289808, 'fraction_numerical': 0.010659560293137908, 'mean_word_length': 5.444206008583691, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Self-Assessments for Workshop Evaluations

There are times in research, as in life, when honest debate leads to common ground. Lam (2009) presents arguments contradicting the conclusions drawn in a 2008 article by D’Eon, Sadownik, Harrison, & Nation, published earlier in this journal. Although we stand by the value of the 2008 study, our own reanalysis of the original data provides additional affirmation of Lam’s (2009) conclusion, but for different reasons. Furthermore, we continue to propose that more research might provide insights into when or if self-assessments would be useful in detecting workshop effectiveness. The article by D’Eon et al. (2008) reports a study to test the hypothesis that individual judgments could provide valuable information when considered in aggregate. It is possible to reduce the error associated with individual judgments, a fundamental property of reliability, thus increasing the utility of the measurements being collected. This is the reason that assessment strategies that use sampling (e.g., Objective Structured Clinical Examinations) do indeed provide useful information in the aggregate even though performance on specific stations fails to predict performance on other specific stations. If we average across numerous selfassessments, we might dilute the error to the point of being able to determine whether learning has taken place for a group, on average, thereby allowing cautious inferences to be made about the effectiveness of an intervention. As self-assessment data are simple and inexpensive to gather, it is vital to know whether education researchers and program evaluators should continue to rely on aggregated self-assessments as meaningful sources of information. The article by D’Eon et al (2008) reported that self-assessments, aggregated across workshop participants, yielded an effect size from pretest to posttest of a magnitude comparable to aggregated external assessments of performance collected across the same interval, despite the fact that the correlation between self-assessments and external judgments was poor. It was concluded, therefore, that the data provided ‘‘reasonably compelling evidence’’ that aggregated self-assessments may generate useful indicators of workshop effectiveness. Lam (2009) has taken exception with these conclusions, arguing that (a) self-assessments are invalid; (b) the observed change in the external assessments may not be attributable to the training workshop; and (c) the findings may not be generalizable to other contexts. We disagree. To argue against the validity of self-assessments at the individual level is immaterial. D’Eon et al did not claim that individual self-assessments were valid and there is ample evidence, including the 2008 study in question, suggestive of the contrary. However, this does not mean that, once aggregated, self-assessments necessarily remain invalid. Second, in any test of construct validity, one examines whether scores on the measure being tested