{'fraction_non_alphanumeric': 0.024640657084188913, 'fraction_numerical': 0.002053388090349076, 'mean_word_length': 5.914893617021277, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Comprehensive Evaluation of an Educational Software Engineering Simulation Environment

Software engineering educational approaches are often evaluated only anecdotally, or in informal pilot studies. We describe a more comprehensive approach to evaluating a software engineering educational technique (SimSE, a graphical, interactive, customizable, game-based software engineering simulation environment). Our method for evaluating SimSE went above and beyond anecdotal experience and approached evaluation from a number of different angles through a family of studies designed to assess SimSE's effectiveness and guide its development. In this paper, we demonstrate the insights and lessons that can be gained when using such a multi-angled evaluation approach. Our hope is that, from this paper, educators will: (1) learn ideas about how to more comprehensively evaluate their own approaches, and (2) be provided with evidence about the educational effectiveness of SimSE.