A Bayesian/Information Theoretic Model of Bias Learning
  In this paper the problem of learning appropriate bias for an environment of
related tasks is examined from a Bayesian perspective. The environment of
related tasks is shown to be naturally modelled by the concept of an {\em
objective} prior distribution. Sampling from the objective prior corresponds to
sampling different learning tasks from the environment. It is argued that for
many common machine learning problems, although we don't know the true
(objective) prior for the problem, we do have some idea of a set of possible
priors to which the true prior belongs. It is shown that under these
circumstances a learner can use Bayesian inference to learn the true prior by
sampling from the objective prior. Bounds are given on the amount of
information required to learn a task when it is simultaneously learnt with
several other tasks. The bounds show that if the learner has little knowledge
of the true prior, and the dimensionality of the true prior is small, then
sampling multiple tasks is highly advantageous.
