{'fraction_non_alphanumeric': 0.008917197452229299, 'fraction_numerical': 0.0, 'mean_word_length': 5.55, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Statistically identifying basic color terms

Basic color terms were originally defined by Berlin and Kay based on linguistic definitions and refer to a subset of color expressions that are universal across languages. In this paper, we investigate and report that basic color terms demonstrate much stronger statistical characteristics that differentiate them from other color words. We introduce a probabilistic interpretation of color naming data, and define the notion of term deviation which measures how much individual speakers’ usages of a color term deviate from the average population. Our analysis of eight languages reveals strong correlation and evidence that basic color terms can be quantitatively characterized as terms whose usages exhibit the least amount of deviation.