Unsupervised Domain Adaptation of Contextual Embeddings for Low-Resource
  Duplicate Question Detection
  Answering questions is a primary goal of many conversational systems or
search products. While most current systems have focused on answering questions
against structured databases or curated knowledge graphs, on-line community
forums or frequently asked questions (FAQ) lists offer an alternative source of
information for question answering systems. Automatic duplicate question
detection (DQD) is the key technology need for question answering systems to
utilize existing online forums like StackExchange. Existing annotations of
duplicate questions in such forums are community-driven, making them sparse or
even completely missing for many domains. Therefore, it is important to
transfer knowledge from related domains and tasks. Recently, contextual
embedding models such as BERT have been outperforming many baselines by
transferring self-supervised information to downstream tasks. In this paper, we
apply BERT to DQD and advance it by unsupervised adaptation to StackExchange
domains using self-supervised learning. We show the effectiveness of this
adaptation for low-resource settings, where little or no training data is
available from the target domain. Our analysis reveals that unsupervised BERT
domain adaptation on even small amounts of data boosts the performance of BERT.
