{'fraction_non_alphanumeric': 0.01521933751119069, 'fraction_numerical': 0.0, 'mean_word_length': 5.425287356321839, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Document Selection for Knowledge Discovery in Texts: Framework Development and Demonstration

The large and constantly growing amounts of available text documents hold great potential for the exploration of knowledge. However, in the light of the vast quantity and variety of available documents, one fact should not be forgotten: the results of a knowledge discovery in texts are only as good as the underlying document collection. That is why analysts have to ensure that document collections adequately represent the specific area under examination and thereby to minimise the bias and to maximise the generalisable nature of the knowledge brought to light. Surprisingly, knowledge management research has barely paid any attention to the problems of such a document quality assessment and rigorous document selection. This paper addresses that research gap and makes two contributions: In the first step, building on a cross-disciplinary exchange with social research, development of a framework for the quality assessment and collection of documents. This artefact provides concrete guidance for compiling sui...