Learning Functions to Study the Benefit of Multitask Learning
  We study and quantify the generalization patterns of multitask learning (MTL)
models for sequence labeling tasks. MTL models are trained to optimize a set of
related tasks jointly. Although multitask learning has achieved improved
performance in some problems, there are also tasks that lose performance when
trained together. These mixed results motivate us to study the factors that
impact the performance of MTL models. We note that theoretical bounds and
convergence rates for MTL models exist, but they rely on strong assumptions
such as task relatedness and the use of balanced datasets. To remedy these
limitations, we propose the creation of a task simulator and the use of
Symbolic Regression to learn expressions relating model performance to possible
factors of influence. For MTL, we study the model performance against the
number of tasks (T), the number of samples per task (n) and the task
relatedness measured by the adjusted mutual information (AMI). In our
experiments, we could empirically find formulas relating model performance with
factors of sqrt(n), sqrt(T), which are equivalent to sound mathematical proofs
in Maurer[2016], and we went beyond by discovering that performance relates to
a factor of sqrt(AMI).
