Multi-Stage Document Ranking with BERT
  The advent of deep neural networks pre-trained via language modeling tasks
has spurred a number of successful applications in natural language processing.
This work explores one such popular model, BERT, in the context of document
ranking. We propose two variants, called monoBERT and duoBERT, that formulate
the ranking problem as pointwise and pairwise classification, respectively.
These two models are arranged in a multi-stage ranking architecture to form an
end-to-end search system. One major advantage of this design is the ability to
trade off quality against latency by controlling the admission of candidates
into each pipeline stage, and by doing so, we are able to find operating points
that offer a good balance between these two competing metrics. On two
large-scale datasets, MS MARCO and TREC CAR, experiments show that our model
produces results that are either at or comparable to the state of the art.
Ablation studies show the contributions of each component and characterize the
latency/quality tradeoff space.
