{'fraction_non_alphanumeric': 0.03507036896158235, 'fraction_numerical': 0.01882845188284519, 'mean_word_length': 4.751695471450448, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A Neural Network-Based Linguistic Similarity Measure for Entrainment in Conversations

Linguistic entrainment is a phenomenon where people tend to mimic each other in conversation. The core instrument to quantify entrainment is a linguistic similarity measure between conversational partners. Most of the current similarity measures are based on bag-of-words approaches that rely on linguistic markers, ignoring the overall language structure and dialogue context. To address this issue, we propose to use a neural network model to perform the similarity measure for entrainment. Our model is context-aware, and it further leverages a novel component to learn the shared high-level linguistic features across dialogues. We first investigate the effectiveness of our novel component. Then we use the model to perform similarity measure in a corpus-based entrainment analysis. We observe promising results for both evaluation tasks.


Introduction
Linguistic entrainment is a phenomenon where individuals unconsciously mimic each other in conversation. There has been a large amount of research that studies this phenomenon in a wide range of linguistic dimensions such as acoustic and prosodic (Levitan et al., 2012;Litman et al., 2016), lexical , and syntactical (Branigan et al., 2000). It has long been an interest of dialogue studies because entrainment has been found to associate with various social outcomes such as group relationship (Yu et al., 2019), positive or negative effect (Nasir et al., 2019), being liked by partners (Levitan et al., 2012), and dialogue success (Kawano et al., 2020;Xu and Reitter, 2017). The characteristics make entrainment a valuable tool to build a human-like dialogue system.
Here we are specifically interested in studying entrainment based on text features. One popular type of study focuses on lexical entrainment. While there are various approaches to quantify lexical entrainment, the core instrument of those approaches is a linguistic similarity measure between conversational partners (Brennan and Clark, 1996;Ward and Litman, 2007;Nenkova et al., 2008;Rahimi et al., 2017;Van der Wege, 2009;Stoyanchev and Stent, 2009). Most current approaches are built upon bag-of-words models that rely heavily on linguistic markers such as function words or high-frequency words (Rahimi et al., 2017;Nenkova et al., 2008;Gonzales et al., 2010;Yu et al., 2019;Pennebaker et al., 2007). However, linguistic markers are insufficient to capture context, irony, sarcasm, or other word semantics (Pennebaker and King, 1999). Sparsity caused by low-level word usage raises reliability concern for this type of measure (Zeldow and McAdams, 1993). While more advanced measures in recent entrainment studies are starting to utilize word representation enriched with semantics such as word embeddings (Nasir et al., 2019), the primary comparison granularity is still single words isolated from the conversation flow.
Therefore we propose an alternate approach using neural networks to perform similarity measures for entrainment calculation. Neural network models are data-driven and are highly self-governing. Using neural network-based models allows us to decouple the entrainment similarity measure from the bag-of-words paradigm. Specifically, input sequences can be represented by high-dimensional vectors embedded with semantic meaning. Beyond word-level information, using sequential architectures such as Long-Short Term Memory Network (LSTM) (Greff et al., 2016), the model can learn structural dependencies among input units at different levels. Feature extraction is also fully automated in neural-based models.
Beyond using a neural network framework, we attempt to learn the high-level linguistic features beyond the inherent text representation of dialogues. The conventional similarity measure for entrainment often leverages high-level linguistic features that can be shared across conversations, such as corpus topics (Rahimi et al., 2017), high-frequency words (Nenkova et al., 2008), and general language style reflected by function words (Gonzales et al., 2010). To simulate this mechanism, we introduce an attention-based architecture to our neural model to generalize high-level linguistic features shared across all input dialogues. These high-level features are supposed to be global and agnostic to the actual content and input forms, leading to a better representation generalization for unstructured data. The architecture is inspired by Global Style Token (GST) that have been previously used in the speech synthesis to generalize speech styles (Wang et al., 2018;. Similar architectures have been adopted in other research area such as machine translation task  and dialogue response matching (Humeau et al., 2019). We don't limit our high-level linguistic features to any specific types. Instead, due to the nature of neural network features learning, the high-level features can describe a comprehensive set of features such as language style, sentence structure, and semantics. We name this component shared stylebook as their parameters are globally shared across all inputs. The "style" in our shared stylebook has a broader definition.
Our ultimate goal is to improve the comprehensiveness of lexical entrainment by using our model to perform the similarity measure. The "lexical entrainment" we refer to in this study has a broader definition beyond lexical.
In this study, we examine 2 specific hypotheses: Hypothesis 1: Leveraging high-level features will aid input representation, leading to a more robust model. Hypothesis 2: Our neural network-based measures will capture a stronger entrainment signal compared to the bag-of-words measures. The results show that both of our hypotheses are positive.

Related Work
Matching Dialogue Response Selection Our model follows neural dialogue response matching frameworks. Matching between the dialogue context and responses is a trendy task in building retrieval-based dialogue systems. The neural network-based models received the most attention in recent years. Early studies focus on single-turn interactions that only considers the dialogue context as a single query by concatenating all previous turns (Yan et al., 2016;Lowe et al., 2015;Wang and Jiang, 2016). Later studies are more interested in learning multi-turn interactions so that the multiple turns in the context are all used as separate queries (Zhou et al., 2018;Lu et al., 2019;Tao et al., 2019). Recent studies show increasing interests in using pre-trained language models such as BERT (Devlin et al., 2019;Wu et al., 2020;Dario Bertero, 2020). Our work focuses on building a single-turn dialogue response matching model. Compared to the existing single-turn model, we add a component to facilitate learning high-level linguistic features.
Style Response Generation/Selection Because we attempt to study the "style" of language, another closely related research topic is dialogue style generation or selection. One typical strategy to generate stylized dialogue responses is to employ 2 separate training stages for response generation and style controlling. Works in style controlling use different approaches such as pre-training stylized language models (Niu and Bansal, 2018), fine-tuning model with styled corpus (Akama et al., 2017), using adversarial training (Zheng et al., 2020), and learning a shared latent space between a response and stylized sentences (Gao et al., 2019). Generating personalized  or emotional responses (Zhou et al., 2017) are also in the same category since they all require controlling some type of style. Our study specifically focuses on dialogue style matching, which has been viewed as a subtask in some style generation models (Luo et al., 2018;Niu and Bansal, 2018). Compared to previous studies, rather than a well-defined style, the style learned by our model is generalizing from the input corpus.
Linguistic Entrainment There has been substantial evidence for entrainment in many linguistic dimensions, such as acoustic-prosodic entrainment (Levitan and Hirschberg, 2011;Levitan et al., 2012;Ward and Litman, 2007), lexical (Brennan and Clark, 1996;Ward and Litman, 2007), and syntactic entrainment (Branigan et al., 2000;Stoyanchev and Stent, 2009;Cleland and Pickering, 2003). To evaluate entrainment, early studies often set experimental conditions or control groups (Brennan and Clark, 1996;Branigan et al., 2000;Garrod and Anderson, 1987). In the later corpus-based studies, evaluations are mostly extrinsic such as comparing entrainment between conversational partners and non-partners (Levitan and Hirschberg, 2011;Rahimi et al., 2017), associating entrainment to other interpersonal behav-iors in dialogue such as group relationships (Yu et al., 2019), positive or negative effects (Nasir et al., 2019), being liked by partners (Levitan et al., 2012), and dialogue success (Kawano et al., 2020). Here, we evaluate our entrainment measures by predicting dialogue success reflected by social outcomes.

Data
In this study, we will focus on a constrained nongoal oriented dataset. The Teams Corpus is a smallscale multiparty spoken dialogue dataset 1 (Litman et al., 2016). It consists of 124 multiparty conversations (62 for Game 1 and 62 for Game 2) elicited from 213 native speakers of American English. Each group of speakers participated in a collaborative game called Forbidden Island. This dataset provides transcriptions and surveys evaluating speaker personality and group relationships. Humans transcribe each inter-pausal unit (IPU). Here we will view each IPU as a conversational turn. We choose this dataset because there are many entrainment-related studies on this dataset using the bag-of-word paradigm to establish a benchmark for entrainment analysis.

Model
Our model is a neural dialogue response matching model. It measures the matching between a dialogue context and a response, and it can be used as a similarity measure for entrainment. We train and evaluate the model following the standard framework of dialogue response matching task defined in the following Section 4.1.

Problem Formalization
Given a dialogue context, response matching models determine whether an utterance is proper as a response. Formally, each train and test example is a triplet (C, R, y) where C is the dialogue context, R is a response, and y is a label indicating whether R is proper for C. Given a dialogue D = u 1 , u 2 , ..., u n where u i is the utterance for i-th turn, we can extract a dialogue context C = u 1 , u 2 , ..., u n−1 , a ground truth response R = u n , and we can randomly sample false responses R from the same corpus. Therefore, we can formulate our task as a binary classification task to determine y ∈ (0, 1) for each (C, R, y) as y = 1 indicating the ground truth. A candidate response is positive 1 https://sites.google.com/site/teamentrainmentstudy/corpus when y = 1 and negative when y = 0. Figure 1 shows 3 input examples.

Model Design
Following the practice in prior works (Zhou et al., 2018;Yan et al., 2016;Wu et al., 2020;Dario Bertero, 2020;Lu et al., 2019;Tao et al., 2019), we train our model with a binary classification objective. We adopted a representation-matching-aggregation framework used in previous works (Zhou et al., 2018;. Figure 2 is the model illustration. Note that state-of-the-art dialogue response matching models are mostly multi-turn models. Our model is single-turn because multi-turn models substantially benefit from learning turn interactions by complicated models. To avoid that and focus on our goal in this study, we choose to follow a simpler single-turn model design.

Representation (Encoder)
Embedding Layer The embedding layer transforms our input of subword tokens to highdimensional continuous representations. Given a dialogue context C and a response candidate R, the representations are C = [e cw 1 ,...e cw n ] and R = [e rw 1 ,...,e rw n ], where e cw i and e rw i represents the embeddings of the i-th token of C and R respectively. Here C ∈ R nc×d and R ∈ R nr×d where n c , n r and d denotes the number of tokens in the context, the number of tokens in the response, and the embedding size, respectively.

Shared StyleBook
The stylebook consists of a set of randomly initialized global key-value pairs. Unlike the self-attention (Vaswani et al., 2017) that the key (K) and value (V) are the linear transformation of input query (Q) itself, our K and V are global for all Q. The stylebook is followed by a multihead scaled dot-product attention (Vaswani et al., 2017) that performs as a similarity metric between the key-value set and the input embeddings. Equa- tions 1 and 2 define the attention function where the query (Q) is the input embeddings of the encoders. Specifically, Q is equivalent to C in the context encoder or R in the response encoder. V denotes value consisted of randomly initialized weights so that V ∈ R T ×dv where T and d denote the size of the stylebook and its dimension. Here we let d be the same as the embedding dimension because we will apply a residual connection later. Key (K) is a linear transformation of V.
For each head i in n heads, we have where n q , T and d i are the query length, the size of the stylebook and the size of each head. The output of the attention layer is a similarity matrix M style ∈ R nq×d where n q = n c for context and n q = n r for the response. We can view this similarity matrix as style embeddings for they represent the contribution of input embeddings on each type of "style" in the stylebook. We employ a residual connection and layer normalization (Add&Norm) after the attention. Thus the final output is a hybrid embedding vector that combines the inherent and style embeddings, which is denoted as C hybrid ∈ R nc×d for the context and R hybrid ∈ R nr×d for the response.
LSTM Layer We choose to use an LSTM to learn the dependencies and temporal relationships between input features. LSTMs are a popular type of RNN to model sequential inputs for its prominent ability to control the short-term or long-term information. In our case, the inputs for this layer are hybrid embeddings from the stylebook, and the outputs are hidden states for each time step denoted by H c ∈ R nc×d h for C , and H r ∈ R nr×d h for R, where d h is the number of hidden units. We will use H c and H r as the final context and response encodings generated from the encoders.

Matching
This layer performs the matching between context and response encodings. We use the scaled dotproduct attention (Vaswani et al., 2017) to measure the similarity between context encodings H c and response encoding H r . Specifically the query Q is the response encoding H r , and the key-value pairs are from context encoding H c . This allows a response to query the most related context information stored in value. Thus, each element in the resulting matrix reflects the similarity between the response and context until the i-th text segment. The layer output is a similarity matrix M r,c , which M r,c ∈ R nr×d .

Aggregation
Similar to prior works in neural matching networks Zhou et al., 2018;Lu et al., 2019), we use an aggregation layer to aggregate matching across segments. Our model aggregates all the segmental matching given by M r,c using an LSTM layer. We use the last hidden state h nr from the aggregation layer as the sequence-level matching.

Projection
The output vector h nr will be fed into a dense layer followed by a softmax layer. The output probability is used as the matching score g between the context C and a response candidate R. Formally, the g is calculated as in Equation 3. The matching score g will be used as the similarity measure in the entrainment task.
where W and b are learned parameters.

Train the Matching Model
We firstly train our matching model on the dataset. We create a Teams Corpus dataset for dialogue response matching task (see Section 4.1). We sampled examples from each dialogue. To make an example, we extract the previous 5 turns as the dialogue context and the following turn as the ground truth responses. This process results in 107,420 positive instances. We split positive instances in train, validation, and test based on a ratio of 6:2:2. Then for each positive instance, we randomly sampled 9 false responses for validation and test sets, and 1 false response for the train set. This operation results in a dataset of 129K, 215K, 215K examples in train, validation, and test set.

Measuring Entrainment as Convergence
Our approach to measure entrainment in Teams Corpus is based on Yu et al. (2019). For each conversation in the corpus, we first split it into 10 equivalent time intervals. For an utterance i in the interval j, we use above model to score the similarity between i and the dialogue context C consisted of the previous 10 turns. Equation 4 shows the calculation. g(C, i) denotes the model generated matching score between context C and i (see Section 4.2.4). Then we average the similarity score over the total n utterances spoken by a speaker during interval j. Yu et al. (2019) use a bag-of-words based similarity score to quantify group difference, and then calculate convergence. Note that the baseline has 2 types of bag-of-words similarity scores depending on different algorithms, but we do not worry about them here because we will replace the bag-of-words score with our neural one. Defined in Equation 5, team difference (TDiff ) is the averaged similarity difference for pair-wise speakers supposing there are m speakers speak in the interval j. Shown in Equation 6, entrainment is measured as the convergence, which indicates the increase in partner similarity, between 2 arbitrary intervals q and p with q being earlier than p.  (2019), we use their group difference and convergence formula, but with a more advanced model-generated similarity score.

Hypothesis 1 (H1)
We first hypothesize that leveraging high-level features will aid input representation, leading to a more robust model in matching dialogue responses. We train 2 models on Teams Corpus to examine this hypothesis: one is our proposed model, and another one is a baseline model with the stylebook removed. Next, we determine whether our model outperforms the baseline model.

Evaluation Metrics
We follow the standard metrics of dialogue response matching task to evaluate Recall@1 (R@1), Recall@2 (R@2), and Recall@5 (R@5). The k in the Recall@k means that the true positive response is among the first k ranked candidates.

Implementation Details
Our model is implemented in Pytorch and trained using 3 GPUs. We use pre-trained English byte pair embeddings (bpemb) from BPEmb (Heinzerling and Strube, 2017). Model configuration is tuned on the validation set. The embedding dimension is 300. The maximum token length is 40 for the context and 20 for the response. The size of the stylebook is set to 500. Encoders are shared between the context and the responses. The LSTM layer in encoders has 1024 hidden units.
The aggregation LSTM layer has 128 hidden units. All multi-head attention used in this model have 4 heads. Models are trained in mini-batches with a size of 128. The learning rate is 0.0001. We use Adam optimizer. The loss function is cross-entropy. We train a maximum of 10 epochs and optimize training at the R@1 on the validation set. To understand whether the stylebook learns any style as we expect, we conduct a study to visualize the style embeddings (see Section 4.2.1) generated from the stylebook. We hand-label 130 utterances from the Teams Corpus with 13 categories of styles based on our intuition. For example, utterances claiming acknowledgement such as "yeah" and "yes" are categorized as Acknowledgement. We collect 10 utterances for each category. Please see the appendix for the description of the 13 categories. We extract the averaged style embeddings of these sequences 2 and then project them into a 3D space using t-distributed stochastic neighbor embedding (t-sne) (Van der Maaten and Hinton, 2008) with a perplexity of 5 and learning rate of 1 3 . Figure 3 shows the results. Each data point in the figure represents an utterance, and it is displayed in the figure by its category in a distinct background color. Figure 3a is a clustering overview that shows 2 major clusters. Figure 3b and 3c are the focus views of the first (Cluster 1) and the second cluster (Cluster 1) respectively. Cluster 1 mainly contains short utterances such as questions, acknowledgments, and questions starting with "What". As a counterpart, Cluster 2 mainly contains long utterances such as long questions consisting of more than 2 short questions. In each cluster, utterances belonging to the same category are more likely to be located closely, indicating that the style embeddings can identify utterances with similar characteristics.

Hypothesis 2 (H2)
We hypothesize that our neural network-based measures will capture a stronger entrainment signal compared to the bag-of-words measures. A recent study (Rahimi and Litman, 2020) on Teams Corpus suggests that more robust entrainment measures carrying stronger signals will lead to a more robust prediction of dialogue outcomes. Thus, we examine this hypothesis with an extrinsic evaluation to predict dialogue success on Teams Corpus.

Baseline Models
The baseline is a bag-of-word approach from Yu et al. (2019) on the Teams Corpus. We only focus on Game 1 as the baseline performance was only published for that portion of the Teams data.

Validate our Measure
Before the prediction, we first validate our similarity measures to ensure they convey some linguistic signals associated with entrainment. Thus we calculate the Pearson correlations between our baseline convergence variables and their baseline counterparts. Note that the baseline approach provides 2 types of convergence variables of being weighted and unweighted based on different bagof-words algorithms. Furthermore, to investigate the impact of the stylebook in our model, we remove the stylebook and examine the correlations again. The results are shown in Table 2. We first find that our Max and absMax are strongly correlated to the baseline convergence variables. This finding suggests that the neural model can be used as a similarity measure for entrainment due to its correlation. On the other hand, the correlation becomes much weaker if we eliminate the stylebook from our model. Intuitively, this finding implies that the stylebook may contribute to capture the linguistic signal related to entrainment.
(a) Clustering overview shows 2 major clusters.
(b) A focus view of Cluster 1, which contains many short sequences.
(c) A focus view of Cluster 2, which contains many long sequences.   (Wendt et al., 2009;Wageman et al., 2005;Guzzo et al., 1993;Gevers et al., 2006). Conflict scales reflect the conflicts in completing tasks, work processes, and interpersonal relationships. IVs are convergence variables described in the Equation 7. We also noticed that the most predictive IV across all 3 models is absMax, which represents the maximum magnitude of convergence. Also, negative entrainment coefficients reveal that a higher convergence signals less conflict in the conversation. This finding is aligned with existing findings.

Case Study
We perform a case study on the similarity score g (see the Equation 3) generated by our model to investigate how it reflects entrainment and whether the shared stylebook has any impact on that. Based on our interpretation, we cherry-pick 2 Teams Corpus dialogue examples that exhibit different types of entrainment. Table 4 shows the dialogue context, the ground truth response, and the scores generated from our neural model when using and not using the shared stylebook. In the first example, similarly to the context, the response contains an exclamation immediately following a phrase starting with "it's". This example can be interpreted as a case of structure entrainment because the same sentence structure is repeated in the response. In the second example, the phrases "this one" and "that one" are frequently used in the context. The response also contains "that one", and more notably, the speaker chooses to say "the lion one" when there is a simpler alternation "the lion". We interpret this example as a phrase entrainment. For the model-generated similarity scores, the neural models, including the model using and not using the stylebook, score more than 0.5 out of 1, indicating that the models likely predict the response as a proper response for the context. Although we expect using the stylebook would aid the model in recognizing entrained cases, the model using the stylebook does not assign higher scores for both of the entrained cases than the model not using the stylebook. This observation is not surprising because neural models, in general, are difficult to interpret. The high-level linguistic features may not align with our intuition.

Conclusion and Future Work
We present a neural dialogue response matching model specifically designed as a similarity measure for lexical entrainment. We propose to leverage the shared stylebook to generalize the high-level shared linguistic features across dialogues. The results suggest that the shared stylebook improves the model performance in a dialogue response matching task. We perform several ablation studies to understand the impact of the stylebook and the underlying meaning of its embeddings. We find our similarity measure is strongly correlated with an existing bag-of-words entrainment measures. Removing the stylebook will weaken the correlation, implying that the stylebook is vital for generating meaningful entrainment measures. We then conduct an extrinsic evaluation to compare our measure and the bag-of-words measure in dialogue outcome prediction. Our measure leads to a more robust prediction model with a stronger entrainment signal. On the other hand, the improvement of prediction is not caused by using the stylebook. The Extrinsic evaluation has its limitation.

Acknowledgement
This study is inspired and partially based upon an unpublished work of Mingzhi Yu during her internship in Microsoft Corporation, Redmond. We thank Microsoft researchers for their valuable feedback.