{'fraction_non_alphanumeric': 0.026473099914602904, 'fraction_numerical': 0.0, 'mean_word_length': 5.659090909090909, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Two-Stream Spatiotemporal Compositional Attention Network for VideoQA

This study tackles video-question answering (VideoQA), which requires spatiotemporal video reasoning. VideoQA aims to return an appropriate answer about textual questions by referring to image frames in video. Based on the observation that multiple entities and their movements in a video can be important clues for deriving correct answers, we propose a two-stream spatiotemporal compositional attention network that achieves sophisticated multi-step spatiotemporal reasoning using both motion and detailed appearance features. In contrast to the existing video reasoning approach that uses frame-level or clip-level appearance and motion features, our method simultaneously attends the region-level appearance features of multiple entities as well as the motion features guided by the attending words in the textual question. Furthermore, it progressively reﬁnes internal representation and infers answers by multiple reasoning steps. We evaluate our method on short- and long-form VideoQA benchmarks, MSVD-QA, MSRVTT-QA, and ActivityNet-QA, and achieve state-of-the-art accuracy on these datasets.