{'fraction_non_alphanumeric': 0.02869360436291522, 'fraction_numerical': 0.010411502231036193, 'mean_word_length': 4.932720588235294, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Hierarchical Attention Network for Evaluating Therapist Empathy in Counseling Session

Counseling typically takes the form of spoken conversation between a therapist and a client. The empathy level expressed by the therapist is considered to be an essential quality factor of counseling outcome. This paper proposes a hierarchical recurrent network combined with two-level attention mechanisms to determine the therapist's empathy level solely from the acoustic features of conversational speech in a counseling session. The experimental results show that the proposed model can achieve an accuracy of 72.1% in classifying the therapist's empathy level as being"high"or"low". It is found that the speech from both the therapist and the client are contributing to predicting the empathy level that is subjectively rated by an expert observer. By analyzing speaker turns assigned with high attention weights, it is observed that 2 to 6 consecutive turns should be considered together to provide useful clues for detecting empathy, and the observer tends to take the whole session into consideration when rating the therapist empathy, instead of relying on a few specific speaker turns.


Introduction
Psychotherapy is typically conducted through conversation between a therapist and a client, aiming to help the client change behavior, overcome difficulties and/or manage distress. In the field of psychotherapy, empathy is described as "the therapist's sensitive ability and willingness to understand the client's thoughts, feelings, and struggles from the client's point of view" [1]. Previous studies have shown that therapist empathy is positively associated with client outcomes and is considered an important quality indicator of psychotherapy [2,3,4,5]. Empathy level of the therapist is often evaluated in a subjective manner, either by the therapist (empathic resonance with the client), the observer (expressed empathy), or the client (received empathy) [3]. However, subjective assessment is time-consuming and costly. The present study aims to investigate methods of automatically analyzing conversational speech in counseling sessions and determining the therapist's empathy level. The outcome of objective evaluation would be useful to the training and improvement of counseling skills.
Spoken language carries a wide range of information, in which one of the major dimensions is determined by and related to the speaker's emotion, desire, intent, and mental state. It was shown that the empathy level expressed by the therapist could be predicted from language cues found in the transcripts of therapy-client interactions [6,7,8,9]. As the primary medium of communication between the therapist and the client, the acoustic speech signal is believed to contain pertinent information about the speaker state. For example, the vocal cues, including pitch, intensity, duration, and speech rate, were investigated in the analysis of therapist empathy [10,11,12,13]. These investigations were focused on the relation between lowlevel speech features and therapist empathy. In this paper, we present a deep neural network (DNN) based system to determine the therapist's empathy level from acoustic properties of speech in the therapist-client conversation.
In a typical counseling session, a therapist and a client take turn to speak, constituting a long conversation. A speaker turn refers to the time period during which only one person speaks. A counseling session may contain hundreds of speaker turns. A speaker turn can be further divided into sub-turns for analysis and feature extraction. In this way, each counseling session is represented by a hierarchical structure, i.e., a group of sub-turns constitute a turn, many speaker turns make up the session. A hierarchical recurrent neural network is adopted to model speech features at different levels of the hierarchy. It is noted that different sub-turns and turns carry different importance weights in determining the therapist empathy. Therefore two levels of attention mechanisms [14,15], one at the sub-turn level and the other at the turn level, are incorporated to improve the performance of the model. To our best knowledge, this is the first use of a hierarchical recurrent neural network combined with twolevel attention mechanisms (HRAN) for evaluating the therapist empathy from conversational speech between the therapist and the client.
Experimental results show that the proposed system is able to determine whether the therapist's empathy level is high or low solely from the speech signals in a given counseling session. It is found that the speech from both the therapist and the client contribute to predicting the third-party observer's subjective rating. By analyzing the speaker turns assigned with high attention weights, it is suggested that consecutive turns should be considered in conjunction, and the observer takes a holistic approach in empathy rating instead of relying on specific parts of the conversation. Section 2 describes a large-scale counseling speech database used in this research. Section 3 introduces the proposed HRAN model for empathy classification in detail. Section 4 presents the experiments and results, followed by discussion and conclusion in Section 5 and Section 6, respectively.

Speech database of counseling
The speech data used in this study are from audio recordings made in the counseling practicum for therapist trainees at the Department of Education Psychology, the Chinese University of Hong Kong (CUHK). Clients of the counseling practicum were adults who sought counseling assistance over various concerns related to emotion, stress, relationship, career, personal growth, and self-esteem. Before the counseling sessions, the trainees discussed the goals and strategies with their supervisors. All therapists and clients spoke Hong Kong Cantonese.
Each counseling session was about 50 minutes long. The therapist's empathy level in each of the counseling sessions was rated by trained raters according to the Therapist Empathy Scale (TES) [16]. TES scores in our data range from 9 to 63, with larger values indicating higher therapist empathy.
The audio recording of a session was divided into speakerturn-based audio segments according to the transcription and speaker turn markers obtained manually. The turn-level speechtext alignment is done automatically as described in [17]. Each speaker turn is then divided into sub-turns, which are separated by major pauses of 0.5 second or longer. In this work, a total of 118 counseling session recordings from 39 different pairs of therapists and clients are used. According to their TES scores, the sessions are divided into two groups. The high-empathy group consists of 61 sessions and the low-empathy group has 57 sessions. TES scores of high-empathy sessions range from 42 to 56.5 (46.34±3.58), and those of low-empathy sessions range from 18 to 36 (30.40±4.79). Table 1 gives a summary of the speech data used in this study.

Proposed system
The proposed model architecture is shown as in Figure 1. Consider a counseling session that comprises M speaker turns, denoted by ti, i = 1, 2, ..., M . The speaker turn ti is divided into Ni sub-turns denoted as xi,j, where j = 1, 2, .., Ni. In the following sub-sections, we will explain how to encode a long counseling session by a high-level embedding learned from acoustic speech features and use this embedding to classify the session as high empathy or low empathy.

From sub-turn speech features to turn-level embedding
Acoustic parameters defined by the eGeMAPS feature set [18] are computed from the speech signal in each sub-turn. eGeMAPS has been applied to numerous paralingusitc tasks, including emotion recognition [18], speech intelligibility prediction [19], and detection of Alzheimer's dementia [20]. Computation of the 88-dimensional eGeMAPS feature vector is implemented by the openSMILE toolkit [21] following the default configuration.
A bidirectional gated recurrent unit (GRU) [22] is used to convert the sub-turn acoustic feature vector into an sub-turnlevel embedding. The bidirectional GRU makes use of contextual information from both forward and backward time directions. The hidden state hi,j for sub-turn xi,j is obtained by concatenating the forward GRU state −→ hi,j, which reads turn ti from xi,1 to xi,N , and the backward GRU state ←− hi,j, which reads from xi,N to xi,1.
To model the non-uniform contributions from individual sub-turns, an attention mechanism is included in the model, A one-layer Multi-layer Perceptron (MLP) is first used to generate ui,j as a hidden representation of hi,j. The importance of the sub-turn is measured as the similarity between ui,j and a sub-turn-level vector ux. A softmax function is applied to compute the normalized importance weight αi,j. Lastly, a turn-level embedding ti is obtained as a weighted sum of subturn GRU hidden states. The learnable vector ux has the same dimension as ui,j and is initialized randomly. It is used as a query over the sub-turns [23].

From turn-level embeddings to session embedding
Given the turn-level embeddings, the session-level embedding is obtained using a similar approach. A bidirectional GRU is used again to obtain the hidden states for the speaker turns: The hidden state hi for turn ti is derived based on the context information from both forward and backward directions. In order to assign different importance weights related to therapist empathy, an attention mechanism is incorporated via a turnlevel vector ut measuring the importance of individual turns, i.e., s is a session-level embedding derived from all turns in the session. Similarly, ut is a learnable vector that can be initialized randomly.
For binary classification of high-low empathy level, the session embedding s is fed into a dense layer, followed by a softmax layer: In our experiments, the training loss is defined as the negative log likelihood of the correct class, where c is the class label of the session k.

Experimental setup
6-fold cross-validation (CV) experiment is arranged by splitting the 118 counseling sessions into three sub-sets, i.e., training (4 folds), development (1 fold) and test (1 fold) set. The input data for empathy level classification are the conversational speech recorded in counseling sessions. The classification output is binary, indicating the therapist empathy level as "high" or "low". There are two speakers in each counseling session. The subturn-level eGeMAPS features from each speaker are normalized with respect to the mean and variance computed from all sub-turn utterances from the speaker in the whole session. The hidden layer sizes of GRU used at the sub-turn and turn levels are 64 and 16, respectively. The query vector in the attention layer has the same dimension as the hidden state of bidirectional GRU. The model is trained for a fixed number of epochs (50) and evaluated on the development set after each epoch. The best model is selected based on the classification accuracy on the development set. The model parameters are updated with the speech data of one session at each iteration. An Adam optimiser with β1 = 0.9, β2 = 0.999 and initial learning rate of 0.0001 is used. The learning rate is reduced by 0.1 every 30 epochs. The average of the classification accuracies on the 6-fold CV is used to indicate the model performance.
In order to evaluate the effectiveness of the two-level attention mechanisms in the model, three baseline models are investigated in the experiment: a hierarchical recurrent neural network without attention (HRN), a hierarchical recurrent neural network with only the sub-turn-level attention (HRSAN) and one with only the turn-level attention (HRTAN). The three models have the same architectures and hyper-parameters as the HRAN, except for the attention layers. Model evaluation is carried out with speech data from either one, or both the therapist and the client.

Results
The classification performance of the proposed HRAN and the three baseline models are shown as in Table 2. Having the attention mechanism appears to be effective and useful. With two-level attention mechanisms, the classification accuracy is greatly improved from 51.8% (no attention) to 72.1%. Turnlevel attention is found to be more effective than sub-turn level attention, but the models with only one level of attention perform much worse than two-level attention. On the other hand, the results suggest that the observer's rating is determined by the speech of both the therapist and the client in the interaction.
It should be noted that the classification results on different test sets in the 6-fold CV have large variation. This may be due to very limited amount of data in our experiments. As a matter of fact, there are no more than 20 sessions in both development and test sets in each fold.  Table 3 gives the confusion matrix on the 118 sessions involved in the CV experiments. There are 85 correctly classified sessions and 33 misclassified sessions. The number of low-empathy sessions misclassified as high-empathy is nearly two times of the misclassified high-empathy sessions. Figure 2 plots the TES scores in ascending order. It is noted that most of the misclassified low-empathy sessions have the scores over 30, approaching the upper bound of the low-empathy group. Likewise, more than half of the misclassified high-empathy sessions have the scores below 44, which is very close to the lower bound of the high-empathy group. These sessions could be viewed as with "moderate" empathy level. The attention weights assigned to individual speaker turns are analyzed to help us better understand how therapist empathy is realized and assessed in the counseling sessions. Speaker turns with attention weight values over a threshold are defined as "important turns". This threshold is set to be the 70th percentile of the attention weights of all turns in a session. The important turns in each of the 118 sessions are extracted. Knowing that speaker interaction is an important aspect of conversation, the neighboring turns of each speaker turn are also considered. The number of important turns in neighborhood is counted. The average number of consecutive important turns in the session is computed. As shown in Figure 3, there are often 2 to 6 consecutive turns classified important turns in a session. This suggests that empathy assessment or detection is not done on isolated or disconnected speaker turns, but involves a few consecutive turns of interaction in the conversation.

Discussion
On the other hand, the distribution of important turns over the time course of a session is investigated. The session is divided into five sections that contain equal number of speaker turns. The percentage ratio of important turns in each section is computed. For each section, the average ratio across 118 sessions is computed. The percentage ratios of important turns in the five sections of counseling session are plotted as in Figure 4. It is noted that important turns tend to be more evenly distributed in the correctly classified sessions than in the misclassified sessions. This suggests that the observer might have taken a holistic approach to determine the therapist's empathy level, rather than focus on a few specific parts of the conversation.

Conclusions
In this work, we propose to use the HRAN to determine the therapist's empathy level from conversational speech in counseling session. Benefiting from the two-level attention mechanisms, the model can achieve an accuracy of 72.1% in classifying a counseling session as "high" or "low" empathy level, solely based on acoustic speech features. The results show that the speech from both the therapist and the client contribute to the observer's rating. By analyzing important turns with high attention weights, it is found that consecutive speaker turns jointly contribute to determining the empathy level. Analyzing the distribution of important turns over a session suggests that subjective empathy rating tends to take the whole session into consideration.