All Sparse PCA Models Are Wrong, But Some Are Useful. Part I:
  Computation of Scores, Residuals and Explained Variance
  Sparse Principal Component Analysis (sPCA) is a popular matrix factorization
approach based on Principal Component Analysis (PCA) that combines variance
maximization and sparsity with the ultimate goal of improving data
interpretation. When moving from PCA to sPCA, there are a number of
implications that the practitioner needs to be aware of. A relevant one is that
scores and loadings in sPCA may not be orthogonal. For this reason, the
traditional way of computing scores, residuals and variance explained that is
used in the classical PCA cannot directly be applied to sPCA models. This also
affects how sPCA components should be visualized. In this paper we illustrate
this problem both theoretically and numerically using simulations for several
state-of-the-art sPCA algorithms, and provide proper computation of the
different elements mentioned. We show that sPCA approaches present disparate
and limited performance when modeling noise-free, sparse data. In a follow-up
paper, we discuss the theoretical properties that lead to this problem.
