Exploring Dual Encoder Architectures for Question Answering
  Dual encoders have been used for question-answering (QA) and information
retrieval (IR) tasks with good results. There are two major types of dual
encoders, Siamese Dual Encoders (SDE), with parameters shared across two
encoders, and Asymmetric Dual Encoder (ADE), with two distinctly parameterized
encoders. In this work, we explore the dual encoder architectures for QA
retrieval tasks. By evaluating on MS MARCO and the MultiReQA benchmark, we show
that SDE performs significantly better than ADE. We further propose three
different improved versions of ADEs. Based on the evaluation of QA retrieval
tasks and direct analysis of the embeddings, we demonstrate that sharing
parameters in projection layers would enable ADEs to perform competitively with
SDEs.
