High-Dimensional Multi-Task Averaging and Application to Kernel Mean
  Embedding
  We propose an improved estimator for the multi-task averaging problem, whose
goal is the joint estimation of the means of multiple distributions using
separate, independent data sets. The naive approach is to take the empirical
mean of each data set individually, whereas the proposed method exploits
similarities between tasks, without any related information being known in
advance. First, for each data set, similar or neighboring means are determined
from the data by multiple testing. Then each naive estimator is shrunk towards
the local average of its neighbors. We prove theoretically that this approach
provides a reduction in mean squared error. This improvement can be significant
when the dimension of the input space is large, demonstrating a "blessing of
dimensionality" phenomenon. An application of this approach is the estimation
of multiple kernel mean embeddings, which plays an important role in many
modern applications. The theoretical results are verified on artificial and
real world data.
