{'fraction_non_alphanumeric': 0.02218430034129693, 'fraction_numerical': 0.0034129692832764505, 'mean_word_length': 5.141361256544503, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Peering Through a Glass Darkly: Integrative evaluation of an on-line course

In this study we describe a wide -spectrum approach to the integrative evaluation of an innovative introductory course in computing. Since both the syllabus, designed in consultation with industry, and the method of presentation of study materials are new, the course requires close scrutiny. It is presented in the distance mode to a class of around 5,000 students and uses a full range of media: paper, broadcast television, interactive CD-ROM, a Web -oriented programming environment, a Web site and computer conferencing. The evaluation began with developmental testing whilst the course was in production, and then used web-based and paper-based questionnaires once the course was running. Other sources of data, in the form of observation of computing conferences and an instrumented version of the Smalltalk programming environment, also provide insight into students’ views and behaviour. This paper discusses the ways in which the evaluation study was conducted and lessons we learnt in the process of integrating all the information at our disposal to satisfy a number of stakeholders.