On Low-rank Trace Regression under General Sampling Distribution
  In this paper, we study the trace regression when a matrix of parameters B*
is estimated via convex relaxation of a rank-penalized regression or via
non-convex optimization. It is known that these estimators satisfy near-optimal
error bounds under assumptions on rank, coherence, or spikiness of B*. We start
by introducing a general notion of spikiness for B* that provides a generic
recipe to prove restricted strong convexity for the sampling operator of the
trace regression and obtain near-optimal and non-asymptotic error bounds for
the estimation error. Similar to the existing literature, these results require
the penalty parameter to be above a certain theory-inspired threshold that
depends on the observation noise and the sampling operator which may be unknown
in practice. Next, we extend the error bounds to the cases when the
regularization parameter is chosen via cross-validation. This result is
significant in that existing theoretical results on cross-validated estimators
do not apply to our setting since the estimators we study are not known to
satisfy their required notion of stability. Finally, using simulations on
synthetic and real data, we show that the cross-validated estimator selects a
nearly-optimal penalty parameter and outperforms the theory-inspired approach
of selecting the parameter.
