Local and Global Context-Based Pairwise Models for Sentence Ordering
  Sentence Ordering refers to the task of rearranging a set of sentences into
the appropriate coherent order. For this task, most previous approaches have
explored global context-based end-to-end methods using Sequence Generation
techniques. In this paper, we put forward a set of robust local and global
context-based pairwise ordering strategies, leveraging which our prediction
strategies outperform all previous works in this domain. Our proposed encoding
method utilizes the paragraph's rich global contextual information to predict
the pairwise order using novel transformer architectures. Analysis of the two
proposed decoding strategies helps better explain error propagation in pairwise
models. This approach is the most accurate pure pairwise model and our encoding
strategy also significantly improves the performance of other recent approaches
that use pairwise models, including the previous state-of-the-art,
demonstrating the research novelty and generalizability of this work.
Additionally, we show how the pre-training task for ALBERT helps it to
significantly outperform BERT, despite having considerably lesser parameters.
The extensive experimental results, architectural analysis and ablation studies
demonstrate the effectiveness and superiority of the proposed models compared
to the previous state-of-the-art, besides providing a much better understanding
of the functioning of pairwise models.
