{'fraction_non_alphanumeric': 0.017517517517517518, 'fraction_numerical': 0.002002002002002002, 'mean_word_length': 5.094512195121951, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Active learning to rank for assessing the linguistic quality of sentences

The traditional way of teaching a foreign language involves long-standing practice and receiving feedback under the guidance of a teacher. However, a large part of the coaching process does not necessarily require the employment of a human teacher, but rather can be automated to facilitate self-practice and to increase the scale of foreign language education by teaching a large number of students without requiring a correspondingly large educational staff. Consequently, automated tutoring and assessment systems for non-native speakers of English are becoming increasingly important. Andersen et al. [2013] present a novel approach to this, in employing a discriminative ranking model as a feedback system on the linguistic quality of individual sentences, in addition to feedback on the quality of an entire text. By its non-specific identification of problematic parts of the text, this can have pedagogic advantages, e.g. by giving the learner the opportunity to come up with a better version himself. We extend their work by using active learning to rank, which aims to reduce the amount of training data required by a ranking model to achieve a certain level of performance by selectively choosing the most informative training instances to annotate and train upon. Annotated data is a major problem in this setting, since there are no publically available datasets with sentence-level annotations, and creating one is time-consuming and expensive. We investigate a number of active selection techniques that are appropriate for a ranking model and systematically compare them to a random learner. More specifically, we demonstrate that active learning can reduce the amount of labeled training data. Moreover, we perform a detailed analysis of the results obtained. To the best of our knowledge, this is the first attempt to investigate the effectiveness of active learning to rank in the domain of learner data.