{'fraction_non_alphanumeric': 0.033572183719017976, 'fraction_numerical': 0.008387172559614707, 'mean_word_length': 4.980050576004496, 'pattern_counts': {'":': 0, '<': 2, '<?xml version=': 0, '>': 2, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Exemplars-guided Empathetic Response Generation Controlled by the Elements of Human Communication

The majority of existing methods for empathetic response generation rely on the emotion of the context to generate empathetic responses. However, empathy is much more than generating responses with an appropriate emotion. It also often entails subtle expressions of understanding and personal resonance with the situation of the other interlocutor. Unfortunately, such qualities are difficult to quantify and the datasets lack the relevant annotations. To address this issue, in this paper we propose an approach that relies on exemplars to cue the generative model on fine stylistic properties that signal empathy to the interlocutor. To this end, we employ dense passage retrieval to extract relevant exemplary responses from the training set. Three elements of human communication -- emotional presence, interpretation, and exploration, and sentiment are additionally introduced using synthetic labels to guide the generation towards empathy. The human evaluation is also extended by these elements of human communication. We empirically show that these approaches yield significant improvements in empathetic response quality in terms of both automated and human-evaluated metrics. The implementation is available at https://github.com/declare-lab/exemplary-empathy.


INTRODUCTION
Empathy is a very human characteristic often defined as the ability to understand and emotionally comprehend the feelings of others intimately. An entire field of research has grown around empathy that delves into its fundamentals [39]. Progress has been made into understanding empathy from the angle of affect [40], personality traits, and demographics [8,9,19]. Understanding of empathy has widespread applications in healthcare, such as in psychotherapy [2] and in general care improvement [27], among others.
Owing to its intrinsic influence on conversations, empathy has garnered much interest in the NLP community. Recent works on empathetic response generation [23,24,26,32]

LEMPEx (uses exemplars)
That's awful. Is he okay? Does it have someone you can do that.

T5-GloVe
That is good to hear. How did he die? MIME Figure 1: Influence of exemplars on the response. and the response. Although response emotion is a key element of empathy, it is far from the complete picture. In fact, the very definition of empathy, which involves visceral understanding of the feeling of other people, is quite subjective and vague and, as such, is quite unwieldy for the empathetic response generation task.
To sidestep this issue, we adopt the idea of exemplars [5] for controllable text generation, where a set of sample responses from the training set, semantically related to the input context, is retrieved (using fine-tuned dense passage retrieval (DPR) [17] model) and fed to the response generator as templates [4]. These template responses guide the generator with stylistic and thematic cues on the response that are perceived as empathetic to the user. This is similar to providing hints for solving a difficult mathematical problem to significantly narrow down the search space. Similarly, we surmise that the exemplars restrict the search space for the decoder.
Our model is called LEMPEx: Learning to EMPathize using Exemplars. Fig. 1 illustrates an instance where the support from exemplars leads to an empathetically resonant and thematically rich response from our model, LEMPEx. The phrase "Oh that sounds scary!" is emotionally appropriate to communicate strong empathy. Moreover, the response makes a thematically appropriate query with "Did he say what was going on?". Based on the similarity between the extracted exemplars and the generated response, we posit that the exemplars helped LEMPEx regulate the nature of the query and emotional intensity. The T5-GloVe model, on the other hand, does not use exemplars. It is probably this what leads it to a partly generic and partly unintelligible query: "Is he okay? Does it have someone you can do that.". "That's awful" phrase is not quite empathetic in this situation; rather, it might be construed as an overreaction. Finally, MIME also misinterprets the situation and asks an inappropriate question. We delve deeper into this in Section 5.2.1.
To further regulate the response generation, we introduce four auxiliary losses that quantify four important aspects related to empathy: emotional presence, interpretation, exploration, and sentiment. Many research works [34,35,46] identify the former three as key aspects of empathy and human communication in general. The role of adequate emotional presence is to form an emotional bond with the user. In Fig. 1, the response phrase "Oh that sounds scary!" is meant to achieve this. Then, correct interpretation is key in regulating emotion and generating context-appropriate comments and queries that exude empathy. Exploration through relevant and appropriate queries signals understanding and interest of the empathetic agent to the user, which is perceived as empathy by the user. The phrase "Did he say what was going on? Is he alright?" is meant to communicate these two attributes. Finally, inspired by the idea of emotion grouping-these groups essentially represent positive and negative sentiment-by Majumder et al. [26], we also add sentiment as an extension of emotional presence.
To this end, we generate synthetic labels for emotional presence, interpretation, and exploration attributes for the gold responses by training classifiers for them with the EmpathyMentalHealth [35] dataset. For sentiment, we get scores from VADER [15]. Subsequently, these labels are predicted from the decoder hidden states to prime the decoder to generate responses with these four attributes at a suitable level. These attributes may be viewed as an external signal to guide the generation process.
To gauge the empathetic quality of the generated responses at a deeper level, we also obtain human evaluation on emotional presence, interpretation, and exploration attributes. This gives us better sense of the generation quality at a more fundamental level. To the best of our knowledge, we are the first to delve into such metrics to assess empathy in generated responses.
Evaluation of empathetic responses is reliant on human evaluation which is resource consuming. To ameliorate this, we repurpose the aforementioned synthetic-label predictors for the empathyrelated attributes emotional presence, interpretation, exploration, and sentiment as automatic evaluators on corresponding attributes. The generated response is fed to the these predictors and the resultant label is compared to the synthetic label of the gold response to obtain macro-F1 and mean absolute error (MAE) scores. In our experiments, we observed good correlation with the other human evaluated metrics across all the discussed models that suggests the reliability of these automated metrics.
Later on in this paper, we empirically show that the combination of these approaches yields strong empathetic responses, outshining the state-of-the-art EmpDG [23] and other baselines.
In a nutshell, the contribution of this paper is three-fold: (1) We mine exemplary responses from the training set, using DPR, to guide the response generation; (2) Empathy-inducing attributes, namely, emotional presence, interpretation, exploration, and sentiment, are introduced as regularizer losses by the means of synthetic labels. (3) We conduct a thorough human and automatic evaluation based on the these attributes, for better understanding of the empathy generation capability of models.

BACKGROUND
Text-To-Text-Transfer-Transformer (T5). Our model is based on the popular T5 model. The Text-to-text-transfer-transformer (T5) is an encoder-decoder based sequence-to-sequence transformer model [30]. In the T5 framework, every natural language processing task, including question answering, language translation, text classification, is framed as a sequence-to-sequence text generation problem. This formulation enables the use of identical architecture and loss functions without any changes across a diverse set of tasks.
T5 follows the self-attention-based encoder-decoder transformer [43] with slight modifications in the architecture and a different position embedding strategy. As opposed to using a fixed sinusoidal position embedding as the original transformer model, T5 uses relative position embeddings that are learned based on the offset between key and query vector. This relative position embedding has been found to work better in practice [14,36]. T5 is pre-trained on the Colossal Clean Crawled Corpus (C4) dataset. A modified masked language modeling (MLM) objective function is used to pre-train the model.
Retrieval-Based Generation. Typically, generative models in NLP employ auto-regressive decoders to generate sentences. However, recent works have shown that simply relying on the model's parameters might be insufficient for specific tasks and might also lead towards undesired effects like hallucinations [38]. In particular, within the literature of controlled text generation, there is increasing focus to provide these models with additional support either through non-parametric access to retrieved content as inputs or through auxiliary losses. [13] proposed a novel strategy that performed language modeling using retrieving fluent responses from human sentences in training data and then editing them. Inspired by this, Weston et al. [47] proposed a retrieve-then-refine framework that involved retrieving relevant responses for a dialog context and then appending these retrievals along with the dialog context to train the overall generator. This simple idea demonstrated good performance to the otherwise daunting task of response generation. Similar approaches were also explored in [4] where the retrieval process was designed to ensure both textual similarity and topical match with the generated response.
While these approaches have been studied in generative applications like language modeling, open-domain dialog, or style transfer, to the best of our knowledge, we are the first to perform a systematic study of this paradigm for empathetic response generation. As we show in the results, our exemplar-guided generation for empathy becomes a better empathetic generator and a simpler solution than previous works in this application.
Related Works in Text Generation. Lately, significant strides have been made in open-domain conversational models [33,44,49]. Models for generating persona-consistent [51] and varied [3] responses have been developed. Their responses are not necessarily empathetic. One aspect of generating such responses involves regulating emotions and sentiments [1,10,48]. GAN-based [11] model for emotion-specific generation has been proposed by Wang and Wan [45]. Zhou and Wang [52] consider emojis as emotion labels. They have also proposed an attention-based [25] Seq-to-Seq [42] model with Conditional Variational Autoencoder [41] for emotion-specific response generation. They, however, require emotion prompt for response generation which may not necessarily be empathetic. Similarly, Wu and Wu [50]'s dual-decoder network responds with input sentiment. Shin et al. [37] devise a reinforcement learning formulation that maximizes the user's sentiment on the generated response. Only recently, following the construction of EmpatheticDialogues [32] dataset, work [23,24,26] has been done on emotion prompt-free automated empathetic response generation. However, these works heavily rely on the emotion labels of the context, which does not capture the entire picture of empathy. Therefore, we leverage exemplar-based generation that has been quite successful in many controlled text generation [5] tasks, such as paraphrasing [12,20] or controlled translation [6]. In this context, previous work [32] has also explored retrieval-based response selection. However, we are the first to explore joint retrieval and generative models, with the former guiding the latter.

METHOD
The existing methods [23,24,26] are strongly reliant on context emotion to generate empathetic responses. However, such approaches overlook the subtle expressions of understanding and personal resonance with the situation of the other interlocutor. To alleviate this, we take two major steps: (1) We use exemplars to cue the generation on stylistic properties that signal empathy to the interlocutor (2) We imbue the responses with four properties, namely sentiment, emotional presence, interpretation, and exploration, that are indicative of empathy [34].
The exemplars for the former step are obtained by dense passage retrieval (DPR) [17] that extracts relevant exemplary responses from the training set for a given context. For the latter step, we firstly train individual classifiers that identify those four properties in a response. During the generative training the generated response is fed to these four classifiers. The four classification losses, based on synthetic labels, are back-propagated through these classifiers to train the generative encoder-decoder model. The resultant model, named LEMPEx, is visualized in Fig. 2.

Task Definition
We are provided with a dialog between a user and an dialog agent, where the agents's role is to provide empathetic responses to the user's statements. Each turn in the dialog alternates between the user and the agent, which forms a sequence of utterances. For such utterances [ 1 , 2 , . . . , ] with the user emoting an emotion , the goal of the agent (our model) is to empathetically respond to the last utterance from the user. In the mentioned sequence, all the odd-indexed { 1 , 3 , . . . , } and even-indexed utterances { 2 , 4 , . . . , −1 } are uttered by the user and the empathetic agent, respectively. Thus for any instance, is always odd, i.e., the last turn in the provided dialog context is always that of the user. The goal of the task is to generate an appropriate and empathetic response-as the + 1 ℎ turn +1 -to this provided context.

Encoder-Decoder Transformer
The recent success of transfer learning from large self-supervised pre-trained models to myriad of NLP tasks had spurred us on to use text-to-text transfer transformer (T5) [31] as the basis of the encoder-decoder setup of our generative model. However, finetuning this pre-trained T5 model on this task counter-intuitively led to poor empathy in the responses. Hence, we resort to train the T5 model from scratch with GloVe [29] word embeddings. We posit that this may be a side-effect of T5 having been pre-trained with plethora of text from the internet that are not particularly empathetic. Therefore, nudging such network to generate empathetic text may require larger number of samples. In Section 5, we empirically further explore this.
To this end, we first prepare the context by concatenating the is the th word in the th utterance. The context is represented as the sum of word ( ) and speaker embeddings ( ): , is the number of words in , and is the embedding length of the words and speakers. The speaker embeddings are trained to represent the speakers of the utterances.
Context ( ) is passed through the T5 encoder (T5 ) and subsequently through T5 decoder (T5 ) for empathetic response generation: where ∈ R × is the encoder output holding context representation, is the generated response of length ( 0 is <start> token), P ∈ [0, 1] ×| | holds the probability distribution on each response token, and is the vocabulary.
In this work, we add add additional modules to this base encoderdecoder T5 transformer network to accommodate retrieved exemplars (Section 3.3) and four auxiliary losses based on sentiment, emotional presence, interpretation, and exploration (Section 3.4).

Exemplar Extraction and Injection
For each input context , we fetch a list of possible responses from the training samples that are likely to embody context-specific empathetic characteristics. The key idea here is to reduce the reliance on emotions for empathetic response generation. Instead,

Response
Training Set

Exemplar Injection
Wow! Have you been to many places?
Oh where are you going? Is it a long trip?
Thats great. Where did you go. we retrieve and show the model possible set of responses, called exemplars, for guidance. Needless to say, these responses are not fully aligned to the context, but they provided important cues towards generating the final response. For example, the exemplars can provide a filtered set of candidate responses with the appropriate empathetic phrases like I am sorry to hear that if the user is sad about an event. We retrieve these exemplars using the popular DPR algorithm that we discuss next.

T5 Decoder
Dense Passage Retrieval. Dense Passage Retrieval (DPR) is a recent algorithm proposed for retrieval in a low-dimensional and continuous space [17]. Unlike traditional retrieval mechanisms, it allows to effectively retrieve samples by matching the embedded representations over a large space of candidates. DPR is a bi-encoder that encodes text passages into a continuous embedding using an encoder such as BERT [7]. Given a query, in our case, and a set of candidate responses from the training set Ψ's, the goal of DPR is to learn a probability distribution (Ψ| ) according the some notion of similarity. This distribution is calculated as follows: where () and () are the respective encoders of the context and candidate exemplars. Given this distribution, we can now choose the top-exemplars based on (Ψ| ). For this, DPR uses the Maximum Inner Product Search (MIPS) method-a sub-linear time approximated algorithm for nearest-neighbour search [16]. We experiment with both the pre-trained DPR that is trained to retrieve answers for questions and also fine-tune DPR to our task. Section 3.5.1 details the training process of the DPR.
Injecting Exemplars. We choose top-responses from the DPR as the exemplars for context . To capture the necessary stylistic attributes, akin to the input context, each extracted exemplar (Ψ ) is encoded with a T5 encoder (T5 ) with identical architecture and GloVe word embedding: The token-level exemplar representations Z ∈ R ·× are meanpooled to obtain a vector representation: = mean(Z ) ∈ R . These representations from all the exemplars are aggregated to obtain the final exemplar representation: To fuse information from the input context and the relevant stylistic cues from the extracted exemplars, exemplar representation is concatenated to the input context representation , from Eq. (1), at token level and fed to a fully-connected layer FC of size : Subsequently, this fused representation ∈ R × is fed to the decoder for the response generation: The generative cross-entropy loss is defined as: where is the reference response.

Empathetic Losses on the Response
According to Sharma et al. [34], emotional presence, interpretation, and exploration are important attributes of empathetic responses: • Emotional Presence. Is emotion in the response absent, implicitly present, or explicit present? • Interpretation. Does the response correctly or incorrectly interpret the user? In addition, does the response contain self-experiences? • Exploration. Is there an attempt in the response to explore the user's emotion? no, generically, or specifically.
To imbue the generated response with the aforementioned qualities, corresponding synthetic labels are obtained from the gold responses. To this end, we train classifiers -identical in structure to Eq. (10), but with independent parameters -for each of these three attributes with the EmpathyMentalHealth dataset 1 . In the EmpathyMentalHealth dataset, each of these attributes is further categorized on a scale of 1/2/3 indicating low/mid/high levels. Hence, the classifier for each of these three attributes is trained with a three class classification objective. We provide more details about this training procedure in Section 4.3.2. The gold responses from EmpatheticDialogues are fed to these trained classifiers to obtain the synthetic labels.
Final hidden state of decoder T5 (Eq. (7)) is taken as the generated-response representation˜∈ R × . For each of the four attributes (denoted as *), we consider the starting token vector [0] and fed to a classifier: For sentiment, tanh activation is used instead of softmax in Eq. (10). We use categorical cross-entropy as loss for emotional presence, interpretation, and exploration: , , and , respectively. Mean squared error is used as loss for sentiment: . Alg. 1 depicts the algorithm used to pre-train the classifiers for empathetic losses and how we use them in our end-to-end generative framework i.e., LEMPEx.

Training
We use Adam [18] to optimize the overall loss: where * are hyper-parameters to assign importance to constituent losses. Please check Alg. 4 and Alg. 5 for a brief overview of the algorithms used for training LEMPEx and doing inference using it. All the models were trained for 50 epochs; early-stopping was used with the patience of 5; the learning rate was set to 1e-5. We used a batch size of 8 for the pre-trained models and a batch size of 32 was used for the Glove-based models. was set to 1.0; , , , and were assigned to 0.1. The lower importance of the empathetic losses is noteworthy; assignment of higher 1 https://bit.ly/2Rwy2gx importance resulted in empathetic, yet, vapid and repetitive responses in our experiments. The hyperparameters were fixed using a random search on the validation dataset. DPR is fined-tuned to predict the correct positive sample from the supplied set of responses. It first encodes the context and responses into a continuous space using two separate encoders termed as bi-encoders. The bi-encoder training is then performed by simultaneously maximizing the similarity between the context and the gold response, and minimizing the similarity between the context and the negative responses. Dot product is used as a measure of similarity and the negative log-likelihood of the positive response is optimized as the objective function: where ( , ) = and , , − are vectors corresponding to context, positive response, and negative responses, respectively, encoded by the DPR bi-encoders. The similarity function ( , ) computes the dot-product between the context vector and the (positive or negative) response vector .
We fine-tune the DPR model with our examples starting from the Natural Questions BERT Base checkpoint [17]. This model uses BERT as bi-encoders and has been pretrained on the Natural Questions dataset [21] for open-domain question answering.
During inference for exemplar extraction, however, for each input context a set of responses from the training dialogues of EmpatheticDialogues with the same emotion is fed to DPR. The top responses with the highest DPR-based dot product similarity with the context are chosen as exemplars. Notably, the set of exemplar responses does not contain the gold response or any other response from the same dialogue for the training samples. In other words, exemplar responses are always selected from a different dialogue (but belonging to the same emotion category). Please check Alg. 2 and Alg. 3 for a brief overview of the algorithms used for DPR training and inference.

.1 Dataset
We evaluate our method on EmpatheticDialogues 2 [32] dataset. It contains 24, 850 dyadic dialogues where the interlocutors empathetically engage with each other. The samples consist of a context, that is an excerpt from a conversation, and a response to the last utterance of the context. All the user utterances of a dialogue are annotated with an emotion label out of total 32 emotion labels. The emotion labels are uniformly distributed across the dataset. The dialogues are split by 8:1:1 (training:validation:test) ratio that is defined by the curators of the dataset.

Baseline Methods and State of the Art
MIME [26]. This method aims at systematically mimicking the overall sentiment of the user in the response to generate empathy. To this end, it splits the set of emotions into two groups having positive and negative emotions. The final response is decoded based on an attention-based blend of these two emotion groups.
EmpDG [23]. EmpDG separately considers the semantic and emotional context during encoding. For response decoding, on the other hand, semantic and emotional representations are fused. It further employs semantic and emotion discriminators that minimize the Wasserstein-1 distance between the semantic and emotional aspects of gold and generated response. This is supposed to instill further semantic and emotional perception into the response. This method is considered as the state of the art (SOTA).
T5 [31]. This baseline has two variants: • T5-PT is a fine-tuned version of a checkpoint of T5-small model trained on C4 dataset [31]; • T5-GloVe (LEMPEx w/o exemplars, empathetic losses) is a T5-small model, with GloVe [29] word embeddings, trained from scratch on EmpatheticDialogues dataset. It is considered for fair comparison with the existing GloVe-based methods, such as MIME and EmpDG. Notably, it is equivalent to an ablated LEMPEx without exemplars and empathetic losses.

Evaluation
We employ both automatic and human-based metrics to evaluate and compare our method and the baselines: 4.3.1 Automatic Metrics. Similar to previous works [23,24,26], we consider the following automatic metrics: • BLEU score [28] has long been used to compare generated text against references in language generation tasks. However, it has been shown to bear little correlation with human evaluation in many dialogue generation tasks, including empathetic response generation [24,26]. As such, we just keep it as a reference rather than using as a mode of comparison.
• Perplexity, defined as raised to the power of cross-entropy, is also kept as reference.

Automatic Synthetic Label-Based Evaluation.
We use the EmpathyMentalHealth [35] dataset to create synthetic labels for the responses in Empathetic Dialogues dataset. The EmpathyMentalHealth contains (context, response) pairs with annotated emotional presence, interpretation, exploration labels. Labels for all the dimensions are provided on a scale of 1/2/3 indicating low/mid/high levels. We consider three different classification models for emotional presence, interpretation, and exploration dimensions. We use the pretrained T5 model [30] having only the encoder part as the backbone of the classification models. A linear layer is added on top of the encoders with softmax activation for the three class classification.
The classification models are trained on the (context, response, label) triplets. We concatenate the context and response pair and pass it as input to the T5 encoder model. The label is then classified from the final layer vector corresponding to the starting token <s>.
We use 20% of the annotations in EmpathyMentalHealth dataset as the validation split. Results obtained in this validation set for the classification of emotional presence, interpretation, and exploration are shown in Table 1. Results indicate that the exploration dimension can be predicted most accurately with close to 94% weighted-F1 (W-F1) score. The emotional presence and interpretation labels are more difficult to predict, with scores being close to 83% and 84%, respectively.
We consider the checkpoints obtaining the best results in the validation set, and use them to create the synthetic labels for the datapoints in EmpatheticDialogues. For a given instance, we concatenate the context and reference response , and pass it to the trained models and use the predicted class as the gold synthetic label. Additionally, we create gold sentiment scores of the reference responses using the VADER tool.
For a generated response, we follow the same method to predict the emotional presence, interpretation, exploration classes and the sentiment score. We then perform automatic evaluations based on the gold synthetic labels/scores and predicted labels/scores. For classification, we consider the macro-F1 between gold and predicted labels. For regression, we consider the mean absolute error (mae) between gold and predicted scores.

Human Ratings.
We randomly choose 100 samples from the testing set and employ three human annotators to evaluate the corresponding responses by different methods. We utilize two sets of attributes to be evaluated by human annotators, namely, coarse attributes and finer empathy-based attributes.
The human annotators rate on a integer scale from 1 (worst) to 5 (best) on the three coarse attributes: • Empathy: How empathetic is the response to the user? • Relevance: How relevant is the response to the context and user?
• Fluency: How linguistically intelligible is the response?
As per Sharma et al. [35], the three empathy-based attributes are strongly related to empathy (elaborated in Section 1): • Emotional Presence: 1 -The response has no emotion, or has emotions that are contradictory to user's feeling, or apathy. 2 -The response implicitly alludes to certain feelings of the user. 3 -The response explicitly alludes to certain feelings of the user.
• Interpretation: 1 -The response communicates no understanding or misunderstanding of the user. 2 -The response communicates correct understanding of the user's situation. 3 -The response also mentions of self-experiences of the agent to amplify the agent's understanding of user's situation. • Exploration: 1 -The response makes no attempt to explore the user's situation or the exploration attempt is invalid in the context. 2 -Exploration attempt is generic. 3 -Exploration attempt is specific to the context. These attribute scores across all 100 samples and three annotators are averaged to obtain the overall evaluation of a particular model.

A/B
Testing. In addition to the Likert-based ratings described above, we also perform ranking-based human evaluation. Given a dialog context and two candidate responses from Model A and Model B, we ask three human annotators to chose their preferred response between the two choices. They may choose to vote tie if both responses are deemed equal. In our experiment, this evaluation was performed on the same set of 100 test samples as for human ratings. We apply majority voting to aggregate votes from the three annotators. In case of three distinct votes from three annotators, a fourth annotator is brought in to break the tie. Finally, we calculate the percentage of times model A is preferred (WIN), model B is preferred (LOSS), and A and B tie (TIE).

RESULTS AND DISCUSSIONS 5.1 Generation Performance
As per Table 2, our model LEMPEx outperforms the state of the art (SOTA) on all human-evaluated metrics by a significant margin. Delving deeper, T5-GloVe, a variant of LEMPEx itself, beats the SOTA on all the human-evaluated metrics, save for exploration dimension. Although, the difference is imperceptible there. This might be attributed to the six-layered transformer encoder and decoder in T5 as opposed to EmpDG and MIME that use single-layered transformer encoder and decoder. The strong baselines MIME and EmpDG perform significantly worse than LEMPEx according to both automatic and human evaluation. We also annotated 100 samples of the gold dataset to set up a fair comparative baseline. The results in Table 2 show that all the models are far from matching the results of the gold standard data. As expected, the pre-trained T5 model-T5-PT outperforms all other methods in terms of fluency. LEMPEx further encapsulates the empathy-specific losses and its performance for empathy score as per the human evaluation has surged by a significant margin.
However, this improvement comes at a cost of a slight decrease in the fluency score when the empathy-specific losses are introduced to LEMPEx. LEMPEx also outshines other models in the empathy-based human evaluation. Results suggest that retrieved exemplars and empathy-specific losses have a notable impact on the performance of LEMPEx in this evaluation scheme. According to the human evaluators, LEMPEx excels in picking up the right emotional keyword for empathy, and it produces more relevant contextual words. Apart from attaining comparatively better BLEU and perplexity scores, LEMPEx also obtains significantly better Distinct-1 and Distinct-2 scores which indicate the superiority of LEMPEx to its competitors in generating diverse responses. As explained in Section 4.3.2, we obtained synthetic labels for the empathy-specific attributes and sentiment polarity. Based on this, we formulated our automatic evaluation of the generated responses with respect to these attributes. In particular, we feed the generated responses of each model to the pre-trained classifiers of these attributes to obtain the labels (Section 4.3.2). To obtain the labels for sentiment polarity, we use VADER. The outcome was then compared against the synthetic labels of the gold standard data. Table 5 presents these results which suggest a noteworthy performance improvement by LEMPEx for all the attributes including sentiment polarity of the generated responses. The significant boost in performance can be seen for exploration which positively correlates with the empathy-specific human evaluation results as depicted in Table 2.
To further verify the findings of human and automatic evaluation of Table 2 and Table 5, we have carried out the A/B testing where annotators were asked to directly compare the responses by two models and declare a win, loss, or a tie. As depicted in Table 6, LEMPEx outperforms SOTA and all the baselines by a large margin.

Qualitative Comparison
We present some qualitative examples of the responses generated by the baselines and our proposed model in Table 4. On an average, we observe that LEMPEx is able to achieve 'High' ratings across Relevance, Emotional Presence, and Interpolation dimensions. Even for Exploration, LEMPEx is the model that generates most amount of responses that attempt to explore the user's state further.
In the first example, we can see that LEMPEx's response includes not just appropriate emotional response, but the response is also inquisitive about the user's dad's health. There is also interpretation on the model's part as it tries to pacify the terrified user by telling "I hope it wasn't too bad". In contrast, other baselines, like MIME provides apathetic response by saying "That is good to hear", and also insensitively concludes that the dad died.
We also observe that in general, due to the additional losses on the empathetic dimensions, we observe LEMPEx's responses to be more compositional of these factors. They not just provide an appropriate empathetic phrase, but also try to interpret the situation and ask relevant questions. The baselines, in contrast, often exhibit only one of the attributes with high intensity.  Oh yeah? Why is that?
We just found out the person working in our accounting department has been stealing money from our company for years.
Yikes! That is terrible! What a awful thing to do! I am glad he finally got busted! Wow, for years, huh?
Wow, that is pretty crazy! What did you do?
I would have screamed!  Most importantly, in our qualitative exploration, we also observe one major trend regarding the association of LEMPEx's responses with the emotion of the user. That is, we find that on average, LEMPEx responds appropriately towards the provided emotion and does not end up being apathetic or insensitive (by being out of line). As seen in the table, examples of the user's emotion and LEMPEx's empathetic phrase, such as Terrified→"Oh no", Confident→"That is great", Sentimental→"I'm sorry to hear that", Annoyed→'That sucks", etc., all map appropriately. We believe this accuracy towards the appropriate empathetic phrase is due to the guidance provided by the retrieved exemplars. As the baseline models do not have such guidance, we observe tendencies to respond inappropriately towards a particular emotion.

Impact of DPR.
We analyzed the effect of exemplars retrieved from DPR in Table 7 and found that it can help in two ways: 1) It can pick up highly semantically similar responses that can aid the decoder to pickup correct/relevant word in the response. This is illustrated in the first example of Table 7. Words generated by LEMPEx such as 'confident' can be found in the exemplar as well. The overall theme of the generated response is also semantically very close to that of the exemplar. In contrast, the w/o EXEMPLAR generation is repetitive with respect to the context, and would have been a more appropriate response in an earlier turn of the dialogue. The phrase 'That is awesome' also contradicts with the speaker's feeling of being nervous.
2) It picks up responses that have stylistic similarity to the reference and pragmatically/stylistically appropriate to the context. We refer to the second example in Table 7. 'That's so nice of him to do. ' in the exemplar is stylistically very close to 'That was very kind of them. ' in the reference. At the same time, the exemplar also fits stylistically with the context. The LEMPEx generation also starts with a very similar sentence as the exemplar, indicating stylistic matching. It then generates 'I'm glad that you had a very good neighbor. ', which displays explicit emotional presence. Overall, the LEMPEx generation can be considered to be a perfect empathetic response in this example.  We find similar patterns in the other two examples in Table 7 as well. Exemplars provide key stylistic and semantic information which are elemental for appropriate empathetic response generation. The model without exemplars struggles to find such patterns and finds it challenging to generate proper empathetic responses.
We further analyze the effect of DPR exemplars in Table 3. Here, we illustrate the contrasting nature of exemplars retrieved by two DPR models, where only one has been fine-tuned on the EmpatheticDialogues dataset. The other model is the original DPR checkpoint [17], which has not been fine-tuned any further. We denote this models as the fine-tuned DPR and the pre-trained DPR, respectively.
In Table 3, we notice some fundamental differences in the nature of the retrieved exemplars. Fine-tuned DPR consistently provides longer and more contextually relevant exemplars. The aspect of interpretation and specific exploration can also be noticed more consistently across the exemplars. Giving a compliment-'Whoa, that is really impressive', along with asking a specific question-'How much do you bench now?' suggest contextual relevance, and the presence of the finer-grained attributes of empathy. However, exemplars from the pre-trained DPR are often much shorter and generic in nature. Very short responses such as 'Why?' or 'Ohh nice!' might be considered as an indication of lack of interest and non-empathetic by the listener.
Overall, we find of trend of fine-tuned DPR providing more diverse and empathetic exemplars to be consistent across the entire dataset. The fine-tuning on EmpatheticDialogues ensures semantically more relevant exemplars in multi-turn contexts as well. We randomly sampled 100 instances of (context, response) pairs and found that fine-tuned DPR exemplars are better than pre-trained DPR exemplars in 92 cases.
5.2.2 Impact of Empathetic Losses. As discussed above, empathetic losses have a notable contribution to LEMPEx's performance.
We illustrate this effect in Table 8. In our analysis, our human evaluators have identified several reasons behind this performance improvement: (1) After adding the empathy-specific losses, LEMPEx attains the ability to ask relevant questions to the user, and this attributes to the conspicuous performance improvement by LEMPEx for the exploration. The first example in Table 8 shows one such case where a specific question about the 'puppy' is asked. (2) Due to the inclusion of emotional presence loss in our model, the generated responses bear more accurate and reasonable amount of emotions. In the second example in Table 8, LEMPEx generates a response having words 'surprise', 'lot of fun' which showcases accurate emotional presence with respect to the context. (3) Exemplars play a major role in establishing the theme of the empathetic response, but it still suffers from producing generic responses. Generic responses fit contexts at the cost of poor relevance. Due to the interpretation and exploration losses, LEMPEx generates more relevant, contextual, and on-topic words in the responses. Consequently, this results in overall better empathetic responses having comparatively higher interpretation and exploration scores (see Table 2).

CONCLUSION
We show the efficacy of employing exemplary responses from the training set in guiding the decoder to generate stylistically and thematically apt responses that communicate empathy in the context. Next, we incorporate four empathy-inducing attributes-emotional presence, interpretation, exploration, and sentiment-in a controlled fashion, using synthetic labels, to regulate the responses. Finally, emotional presence, interpretation, and exploration attributes are also manually evaluated to better gauge the empathetic qualities of the generated responses. All these approaches collectively result in an improved framework for empathetic response generation research. LEMPEx, however, does not employ an end-to-end retrieval and generation approach. The infusion of exemplars into the generative model is also rather simple and naïve. These reveal a number of drawbacks of the model as follows: (1) Lack of relevance measurement: LEMPEx relies on DPR for exemplar retrieval which prohibits the relative relevance ranking of the retrieved responses by the generative model with a generative cross-entropy objective. This can make the model sensitive to noise as it has no means to particularly attend to the relevant information present in the exemplars or discard any noise present in the exemplars. One could use a unique training objective, attentions on the exemplars, employ non-trivial more sound fusion techniques that can directly influence decoded output to attain better results. (2) Worse performance for multiturn context: We and our annotators observed that the trained DPR does not perform very well on multiturn context and often returns identical responses to the first utterance in a dialogue. This again calls for an end-to-end retrieval and generation.