Filling the Gap of Utterance-aware and Speaker-aware Representation for
  Multi-turn Dialogue
  A multi-turn dialogue is composed of multiple utterances from two or more
different speaker roles. Thus utterance- and speaker-aware clues are supposed
to be well captured in models. However, in the existing retrieval-based
multi-turn dialogue modeling, the pre-trained language models (PrLMs) as
encoder represent the dialogues coarsely by taking the pairwise dialogue
history and candidate response as a whole, the hierarchical information on
either utterance interrelation or speaker roles coupled in such representations
is not well addressed. In this work, we propose a novel model to fill such a
gap by modeling the effective utterance-aware and speaker-aware representations
entailed in a dialogue history. In detail, we decouple the contextualized word
representations by masking mechanisms in Transformer-based PrLM, making each
word only focus on the words in current utterance, other utterances, two
speaker roles (i.e., utterances of sender and utterances of receiver),
respectively. Experimental results show that our method boosts the strong
ELECTRA baseline substantially in four public benchmark datasets, and achieves
various new state-of-the-art performance over previous methods. A series of
ablation studies are conducted to demonstrate the effectiveness of our method.
