Attentive Task Interaction Network for Multi-Task Learning
  Multitask learning (MTL) has recently gained a lot of popularity as a
learning paradigm that can lead to improved per-task performance while also
using fewer per-task model parameters compared to single task learning. One of
the biggest challenges regarding MTL networks involves how to share features
across tasks. To address this challenge, we propose the Attentive Task
Interaction Network (ATI-Net). ATI-Net employs knowledge distillation of the
latent features for each task, then combines the feature maps to provide
improved contextualized information to the decoder. This novel approach to
introducing knowledge distillation into an attention based multitask network
outperforms state of the art MTL baselines such as the standalone MTAN and
PAD-Net, with roughly the same number of model parameters.
