UmBERTo-MTSA @ AcCompl-It: Improving Complexity and Acceptability
  Prediction with Multi-task Learning on Self-Supervised Annotations
  This work describes a self-supervised data augmentation approach used to
improve learning models' performances when only a moderate amount of labeled
data is available. Multiple copies of the original model are initially trained
on the downstream task. Their predictions are then used to annotate a large set
of unlabeled examples. Finally, multi-task training is performed on the
parallel annotations of the resulting training set, and final scores are
obtained by averaging annotator-specific head predictions. Neural language
models are fine-tuned using this procedure in the context of the AcCompl-it
shared task at EVALITA 2020, obtaining considerable improvements in prediction
quality.
