Grouping effects of sparse CCA models in variable selection
  The sparse canonical correlation analysis (SCCA) is a bi-multivariate
association model that finds sparse linear combinations of two sets of
variables that are maximally correlated with each other. In addition to the
standard SCCA model, a simplified SCCA criterion which maixmizes the
cross-covariance between a pair of canonical variables instead of their
cross-correlation, is widely used in the literature due to its computational
simplicity. However, the behaviors/properties of the solutions of these two
models remain unknown in theory. In this paper, we analyze the grouping effect
of the standard and simplified SCCA models in variable selection. In
high-dimensional settings, the variables often form groups with high
within-group correlation and low between-group correlation. Our theoretical
analysis shows that for grouped variable selection, the simplified SCCA jointly
selects or deselects a group of variables together, while the standard SCCA
randomly selects a few dominant variables from each relevant group of
correlated variables. Empirical results on synthetic data and real imaging
genetics data verify the finding of our theoretical analysis.
