When are the most informative components for inference also the
  principal components?
  Which components of the singular value decomposition of a signal-plus-noise
data matrix are most informative for the inferential task of detecting or
estimating an embedded low-rank signal matrix? Principal component analysis
ascribes greater importance to the components that capture the greatest
variation, i.e., the singular vectors associated with the largest singular
values. This choice is often justified by invoking the Eckart-Young theorem
even though that work addresses the problem of how to best represent a
signal-plus-noise matrix using a low-rank approximation and not how to
best_infer_ the underlying low-rank signal component.
  Here we take a first-principles approach in which we start with a
signal-plus-noise data matrix and show how the spectrum of the noise-only
component governs whether the principal or the middle components of the
singular value decomposition of the data matrix will be the informative
components for inference. Simply put, if the noise spectrum is supported on a
connected interval, in a sense we make precise, then the use of the principal
components is justified. When the noise spectrum is supported on multiple
intervals, then the middle components might be more informative than the
principal components.
  The end result is a proper justification of the use of principal components
in the setting where the noise matrix is i.i.d. Gaussian and the identification
of scenarios, generically involving heterogeneous noise models such as mixtures
of Gaussians, where the middle components might be more informative than the
principal components so that they may be exploited to extract additional
processing gain. Our results show how the blind use of principal components can
lead to suboptimal or even faulty inference because of phase transitions that
separate a regime where the principal components are informative from a regime
where they are uninformative.
