The Singular Value Decomposition, Applications and Beyond
  The singular value decomposition (SVD) is not only a classical theory in
matrix computation and analysis, but also is a powerful tool in machine
learning and modern data analysis. In this tutorial we first study the basic
notion of SVD and then show the central role of SVD in matrices. Using
majorization theory, we consider variational principles of singular values and
eigenvalues. Built on SVD and a theory of symmetric gauge functions, we discuss
unitarily invariant norms, which are then used to formulate general results for
matrix low rank approximation. We study the subdifferentials of unitarily
invariant norms. These results would be potentially useful in many machine
learning problems such as matrix completion and matrix data classification.
Finally, we discuss matrix low rank approximation and its recent developments
such as randomized SVD, approximate matrix multiplication, CUR decomposition,
and Nystrom approximation. Randomized algorithms are important approaches to
large scale SVD as well as fast matrix computations.
