{'fraction_non_alphanumeric': 0.022244691607684528, 'fraction_numerical': 0.0, 'mean_word_length': 5.265822784810126, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Detection of Off-screen Sound Based on Loss Function of Self-supervised Audio-visual Spatialization

This paper presents a method to detect off-screen sounds based on a loss function of self-supervised audio-visual spatialization. Audio source separation is a long-standing problem, and audio-visual source separation has recently attracted attention in the field of acoustic signal processing, image processing and machine learning. Although visual information in a video has important cues for audio source separation, it does not have information about sounds whose sources are not found in the video. The proposed method detects such off-screen sounds by focusing on correspondences between sounds and their sources in the video. For finding the correspondences, we use audio-visual spatialization, which converts mono audio into spatial audio with visual guidance. Experimental results show that the proposed method can achieve high accuracy in the task of off-screen sound detection.