{'fraction_non_alphanumeric': 0.031210191082802548, 'fraction_numerical': 0.0044585987261146496, 'mean_word_length': 5.234126984126984, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Towards Improving Canonical Correlation Analysis for Cross-modal Retrieval

Building correlations for cross-modal retrieval, i.e., image-to-text retrieval and text-to-image retrieval, is a feasible solution to bridge the semantic gap between different modalities. Canonical correlation analysis (CCA) based methods have ever achieved great successes. However, conventional 2-view CCA suffers from three inherent problems: 1) it fails to capture the intra-modal semantic consistency, which is a necessary element to improve the retrieval performance, 2) it is hard to learn the non-linear correlation between different modalities, and 3) there exists problem in similarity measure due to the fact that the latent space learned by CCA is not directly optimized with certain distance measure. To address above problem, in this paper, we propose an improved CCA algorithm (ICCA) from three aspects. First, we propose two effective semantic features based on text features to improve intra-modal semantic consistency. Second, we expand traditional CCA from 2-view to 4-view, and embed 4-view CCA into a progressive framework to alleviate the over-fitting. Our progressive framework combines the training of linear projection and nonlinear hidden layers to ensure that good representations of the input raw data are learned at the output of the network. Third, inspired by large scale similarity learning (LSSL), a similarity metric is proposed to improve the distance measure. Experiments on three publicly data sets demonstrate the effectiveness of the proposed ICCA method.