{'fraction_non_alphanumeric': 0.017082785808147174, 'fraction_numerical': 0.0, 'mean_word_length': 4.88659793814433, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Language and Vision from a Dynamic Perspective.

Language and pictures are traditionally dealt with as two modalities with quite different properties, with different strengths and weaknesses. What I would like to find out is rather what language and vision have in common. In this study, participants were examining a complex picture and describing it simultaneously. Their eye movements were registered during inspectation of the picture with a SMI iView eye tracker and their spoken language descriptions were recorded and analysed. The common denominator here is the way how we focus attention in language and vision. In a dynamic view, both visual scanning of a picture and spoken language description of the picture can be conceived of with the help of a spotlight metaphor: The picture elements described lie in the focus of a spotlight and are embedded in a context (in the periphery). The spotlight then moves to the next picture element which pops up from the periphery and will lie in the focus for a while. Speakers zoom in and out when talking about different aspects and different portions of the picture. Eye movements are governed in a similar way. When scanning the picture visually, observers focus on one area of interest a time and the spotlight moves quickly to another region which is then scanned with the help of visual fixations. The picture becomes 'linearized' through the visual path. What has been formulated so far is, that language and vision proceed in brief spurts, have a focus and a periphery and 'linearize' the picture content in the process of observation perception, conceptualization and simultaneous description. But apart from that, are there any comparable units in both modalities? What are they based on? Which size do they have? What common cognitive patterns underlie our competence in these disparate modes of thought? How is rhythm important for language and vision? The aims of the study are twofold: a) to compare semantic and sequential patterns in the eye movement data and in the spoken language data and b) to find out if there are similar general strategies in the visual scanning and describing of the picture. In my presentation, I will give some preliminary answers to the questions above and verify the following hypotheses: