Understanding the bias-variance tradeoff of Bregman divergences
  This paper builds upon the work of Pfau (2013), which generalized the bias
variance tradeoff to any Bregman divergence loss function. Pfau (2013) showed
that for Bregman divergences, the bias and variances are defined with respect
to a central label, defined as the mean of the label variable, and a central
prediction, of a more complex form. We show that, similarly to the label, the
central prediction can be interpreted as the mean of a random variable, where
the mean operates in a dual space defined by the loss function itself. Viewing
the bias-variance tradeoff through operations taken in dual space, we
subsequently derive several results of interest. In particular, (a) the
variance terms satisfy a generalized law of total variance; (b) if a source of
randomness cannot be controlled, its contribution to the bias and variance has
a closed form; (c) there exist natural ensembling operations in the label and
prediction spaces which reduce the variance and do not affect the bias.
