Fast optimization of common basis for matrix set through Common Singular
  Value Decomposition
  SVD (singular value decomposition) is one of the basic tools of machine
learning, allowing to optimize basis for a given matrix. However, sometimes we
have a set of matrices $\{A_k\}_k$ instead, and would like to optimize a single
common basis for them: find orthogonal matrices $U$, $V$, such that $\{U^T A_k
V\}$ set of matrices is somehow simpler. For example DCT-II is orthonormal
basis of functions commonly used in image/video compression - as discussed
here, this kind of basis can be quickly automatically optimized for a given
dataset. While also discussed gradient descent optimization might be
computationally costly, there is proposed CSVD (common SVD): fast general
approach based on SVD. Specifically, we choose $U$ as built of eigenvectors of
$\sum_i (w_k)^q (A_k A_k^T)^p$ and $V$ of $\sum_k (w_k)^q (A_k^T A_k)^p$, where
$w_k$ are their weights, $p,q>0$ are some chosen powers e.g. 1/2, optionally
with normalization e.g. $A \to A - rc^T$ where $r_i=\sum_j A_{ij}, c_j =\sum_i
A_{ij}$.
