{'fraction_non_alphanumeric': 0.029311001141987057, 'fraction_numerical': 0.011800532927293491, 'mean_word_length': 5.394160583941606, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Grounding the Simulation of Iconic Gestures in Gesture Typology

How are co-speech iconic gestures used to convey visuo-spatial information? We investigate this question, which is still relatively unexplored (Bavelas et al., 2008), with an interdisciplinary methodology combining the empirical study of speech and gesture use, the elaboration of theoretical reconstructions and the formulation of generation models that enable the simulation of such communicative behaviour with virtual humans. In our talk we will focus on two topics, first, how gesture simulation is grounded in empirical gesture typology and second, how gesture simulation can be used methodologically, looping back to the empirical data on which both, simulation and theoretical modelling are based. Regarding current gesture research, the second topic is entirely new. Even traditionally, gesture research and gesture typology are intimately intertwined as one can see from Ekman and Friesen (1969), Kendon (2004) or McNeill (1992). Recently, new options for gesture typology have arisen due to systematically collected and annotated data such as the Bielefeld Speech And Gesture Alignment corpus (SAGA, Lucking et al. (2010)). Corpus-based empirical methods proceed from rated annotations to classification of recurrent structures and ultimately to an investigation of its generalizability supported by statistical investigations (Hahn and Rieser (2010)). Computational simulation opens up new possibilities enriching this set of methods in many ways. Obviously, gesture simulation has its independent goals in endowing virtual agents with human-like expressiveness. In addition, we use it as a methodological device, more specifically for the post-hoc evaluation of decisions made at various levels of the theory construction process, in other words, as a method of Popperian falsification. As an illustration of every aspect of our methodology, we will discuss a church-window-example from the SAGA corpus shown in Figure 1 throughout the talk (restricted to the top of the window). Empirical Study and Theoretical Perspective. Empirical Study and Theoretical Perspective The empirical study slot in Figure 1 shows under (1) the pointed gothic church-window presented in a VR-video film as perceived from an agent called Router. Empirical study slot (2) shows the gesture of the Router reporting his ride through a VR-town to a Follower, especially his drawing of a pointed top. The Router’s gesture modifies his words “these typical uhm church-windows”. Empirical study slot (3) presents information coming from the rated annotation: The gesture is 33