Cross-Domain Multi-Task Learning for Sequential Sentence Classification
  in Research Papers
  Sequential sentence classification deals with the categorisation of sentences
based on their content and context. Applied to scientific texts, it enables the
automatic structuring of research papers and the improvement of academic search
engines. However, previous work has not investigated the potential of transfer
learning for sentence classification across different scientific domains and
the issue of different text structure of full papers and abstracts. In this
paper, we derive seven related research questions and present several
contributions to address them: First, we suggest a novel uniform deep learning
architecture and multi-task learning for cross-domain sequential sentence
classification in scientific texts. Second, we tailor two common transfer
learning methods, sequential transfer learning and multi-task learning, to deal
with the challenges of the given task. Semantic relatedness of tasks is a
prerequisite for successful transfer learning of neural models. Consequently,
our third contribution is an approach to semi-automatically identify
semantically related classes from different annotation schemes and we present
an analysis of four annotation schemes. Comprehensive experimental results
indicate that models, which are trained on datasets from different scientific
domains, benefit from one another when using the proposed multi-task learning
architecture. We also report comparisons with several state-of-the-art
approaches. Our approach outperforms the state of the art on full paper
datasets significantly while being on par for datasets consisting of abstracts.
