Aggregation of Affine Estimators
  We consider the problem of aggregating a general collection of affine
estimators for fixed design regression. Relevant examples include some commonly
used statistical estimators such as least squares, ridge and robust least
squares estimators. Dalalyan and Salmon (2012) have established that, for this
problem, exponentially weighted (EW) model selection aggregation leads to sharp
oracle inequalities in expectation, but similar bounds in deviation were not
previously known. While results indicate that the same aggregation scheme may
not satisfy sharp oracle inequalities with high probability, we prove that a
weaker notion of oracle inequality for EW that holds with high probability.
Moreover, using a generalization of the newly introduced $Q$-aggregation scheme
we also prove sharp oracle inequalities that hold with high probability.
Finally, we apply our results to universal aggregation and show that our
proposed estimator leads simultaneously to all the best known bounds for
aggregation, including $\ell_q$-aggregation, $q \in (0,1)$, with high
probability.
