{'fraction_non_alphanumeric': 0.022427440633245383, 'fraction_numerical': 0.002638522427440633, 'mean_word_length': 5.221311475409836, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
What are we thinking when we grade programs?

This paper reports on a mixed methods study which examines how four experienced instructors approached the grading of a programming problem. Two instructors used a detailed, analytic approach and two instructors employed a holistic approach. One instructor exhibited elements of a primary trait approach. Even though the four instructors used different grading scales and philosophies, their raw scores were highly correlated (Spearman's rho of .81) supporting the conclusion that experienced instructors usually agree on whether a program is 'very good' or 'very bad'. Clearly there is no single right way to grade programs. Further discourse should be encouraged for the benefit of both educators and students.