{'fraction_non_alphanumeric': 0.02062780269058296, 'fraction_numerical': 0.0013452914798206279, 'mean_word_length': 5.466666666666667, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Semantic Annotation and Automated Extraction of Audio-Visual Staging Patterns in Large-Scale Empirical Film Studies

The study of audio-visual rhetorics of affect scientifically analyses the impact of auditory and visual staging patterns on the perception of media productions as well as the conveyed emotions. In the AdA-project, together with film scientists, we aim to follow the hypothesis of TV reports drawing on audio-visual patterns in cinematographic productions to emotionally affect viewers by largescale corpus analysis of TV reports, documentaries and genre-films of the topos “financial crisis”. As it can be observed in the past and current media coverage of the world-wide financial crisis, TV reports often employ highly emotionalizing staging strategies in order to convey a certain message to the audience. This is also true for public broadcasting agencies who claim to adopt journalistic objectivity. This project therefore aims to make transparent this hard to grasp opinion-forming level of audio-visual reporting and follows the hypothesis of audio-visual staging patterns always aiming at coining emotional attitudes. So far, localization and description of these patterns is currently limited to micro-studies due to the involved extremely high manual annotation effort [4]. We therefore pursue two main objectives: 1) creation of a standardized annotation vocabulary to be applied for semantic annotations and 2) semi-automatic classification of audio-visual patterns by training models on manually assembled ground truth annotation data. The annotation vocabulary for empirical film studies and semantic annotations of audio-visual material based on Linked Open Data principles enables the publication, reuse, retrieval, and visualization of results from film-analytical methods. Furthermore, automatic analysis of video streams allows to speed up the process of extracting audio-visual patterns. This paper will focus on describing the semantic data management of the project and the developed vocabulary for fine-grained semantic video annotation. Furthermore, we will give a short outlook on how we aim to integrate machine learning into the process of automatically detecting audio-visual patterns.