{'fraction_non_alphanumeric': 0.014925373134328358, 'fraction_numerical': 0.003109452736318408, 'mean_word_length': 5.02621722846442, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A Meta Search Approach to Locating and Classifying Reading Material for Learners of Nordic Languages

Research in reading as a psychological process indicates that the quality of the processing of target language input can be improved if the reader finds the text personally significant. By providing students with authentic texts in their own subjects, and by allowing them to share the responsibility for their learning, this can be accomplished. This thesis describes the development of Squirrel, a prototype for locating authentic texts in the Nordic languages published on the World Wide Web. It was designed to facilitate the process of producing authentic learning materials for second language learners. The primary target users are teachers and exchange students to Nordic institutions of higher education. A commercial search engine is used for meta searching the Web. Given a HTML document supplied by the user, Squirrel extracts possible query terms from the most frequent content words of the text and from the HTML meta data. The user chooses which terms to use, and the query is sent to the search engine. The returned results are collected and classified according to language and difficulty level. Finally, the user is presented with a list of links to documents that might suit the user’s need. Although evaluation involving real users remains to be done, initial results are encouraging. Recall and precision of the automatic language identifier range from 92-100 percent. The methods chosen for query term extraction and readability testing are well-tested, but must be further evaluated.