A Self-consistent-field Iteration for Orthogonal Canonical Correlation
  Analysis
  We propose an efficient algorithm for solving orthogonal canonical
correlation analysis (OCCA) in the form of trace-fractional structure and
orthogonal linear projections. Even though orthogonality has been widely used
and proved to be a useful criterion for pattern recognition and feature
extraction, existing methods for solving OCCA problem are either numerical
unstable by relying on a deflation scheme, or less efficient by directly using
generic optimization methods. In this paper, we propose an alternating
numerical scheme whose core is the sub-maximization problem in the
trace-fractional form with an orthogonal constraint. A customized
self-consistent-field (SCF) iteration for this sub-maximization problem is
devised. It is proved that the SCF iteration is globally convergent to a KKT
point and that the alternating numerical scheme always converges. We further
formulate a new trace-fractional maximization problem for orthogonal multiset
CCA (OMCCA) and then propose an efficient algorithm with an either Jacobi-style
or Gauss-Seidel-style updating scheme based on the same SCF iteration.
Extensive experiments are conducted to evaluate the proposed algorithms against
existing methods including two real world applications: multi-label
classification and multi-view feature extraction. Experimental results show
that our methods not only perform competitively to or better than baselines but
also are more efficient.
