{'fraction_non_alphanumeric': 0.015353121801432957, 'fraction_numerical': 0.0030706243602865915, 'mean_word_length': 5.150943396226415, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Accurateness evaluation of an IR-based marking engine for mathematics assessment

This paper presents the results of the preliminary evaluation of the accurateness of the automated marking implemented by an IR-based marking engine on free-response mathematics solutions. The development of the marking engine is based on adaptation of techniques in text retrieval process. The engine incorporates our own similarity measure which models a mathematical expression as a multiset of mathematical tokens. The measure evaluates the similarity between the mathematical expression of a worked-out solution and that of a model solution. The degree of correctness of a worked-out solution that has been compared is indicated by the similarity value. The results of the preliminary evaluation show that the IR-based marking engine is able to perform an automated marking of the solutions of the four (4) questions tested with an average of 97% acurateness when compared to human marking.