{'fraction_non_alphanumeric': 0.014846235418875928, 'fraction_numerical': 0.0, 'mean_word_length': 5.890510948905109, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Learning Abstract Concepts from Multi-Modal Data: Since You Probably Can’t See What I Mean

Models that acquire semantic representations from both linguistic and perceptual input outperform linguistic-only models on various NLP tasks. However, this superiority has only been established when learning concrete concepts, which are usually domain specific and also comparatively rare in everyday language. We extend the scope to more widely applicable abstract representations, and present a multi-modal probabilistic language architecture for learning semantic representations for both concrete and abstract concepts. Our model outperforms alternative approaches in combining input from two modalities and also in propagating perceptual information on concrete to more abstract concepts. We discuss the implications of our results both for optimizing the performance of multi-modal models and for theories of abstract conceptual representation.