CLIP4Caption ++: Multi-CLIP for Video Caption
  This report describes our solution to the VALUE Challenge 2021 in the
captioning task. Our solution, named CLIP4Caption++, is built on
X-Linear/X-Transformer, which is an advanced model with encoder-decoder
architecture. We make the following improvements on the proposed
CLIP4Caption++: We employ an advanced encoder-decoder model architecture
X-Transformer as our main framework and make the following improvements: 1) we
utilize three strong pre-trained CLIP models to extract the text-related
appearance visual features. 2) we adopt the TSN sampling strategy for data
enhancement. 3) we involve the video subtitle information to provide richer
semantic information. 3) we introduce the subtitle information, which fuses
with the visual features as guidance. 4) we design word-level and
sentence-level ensemble strategies. Our proposed method achieves 86.5, 148.4,
64.5 CIDEr scores on VATEX, YC2C, and TVC datasets, respectively, which shows
the superior performance of our proposed CLIP4Caption++ on all three datasets.
