{'fraction_non_alphanumeric': 0.016985138004246284, 'fraction_numerical': 0.0, 'mean_word_length': 5.594405594405594, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Evaluating Research on Software Design and Productivity

A key barrier to achieving the necessary breakthroughs in software design and productivity is that, as a community, we do not have an adequate understanding and consensus on how to evaluate research on the topic. This has hindered publication and funding of research in the area, pushing researchers to explore other topics. Software design and productivity involves, at its core, human creativity and understanding. Rigorous experimental studies of software design and productivity are therefore difficult, expensive and rare. Requiring them as a prerequisite for research in the field would constrict the field in ways that would not serve our interests. So we need to develop benchmarks, challenge problems, frameworks and (partial) consensus to allow evaluation of useful research on software design and productivity without requiring prohibitive experiments involving programmers.