Dynamically-Scaled Deep Canonical Correlation Analysis
  Canonical Correlation Analysis (CCA) is a method for feature extraction of
two views by finding maximally correlated linear projections of them. Several
variants of CCA have been introduced in the literature, in particular, variants
based on deep neural networks for learning highly correlated nonlinear
transformations of two views. As these models are parameterized conventionally,
their learnable parameters remain independent of the inputs after the training
process, which may limit their capacity for learning highly correlated
representations. We introduce a novel dynamic scaling method for training an
input-dependent canonical correlation model. In our deep-CCA models, the
parameters of the last layer are scaled by a second neural network that is
conditioned on the model's input, resulting in a parameterization that is
dependent on the input samples. We evaluate our model on multiple datasets and
demonstrate that the learned representations are more correlated in comparison
to the conventionally-parameterized CCA-based models and also obtain preferable
retrieval results. Our code is available at
https://github.com/tomerfr/DynamicallyScaledDeepCCA.
