{'fraction_non_alphanumeric': 0.026884621488652595, 'fraction_numerical': 0.007141723535946675, 'mean_word_length': 5.099903194578896, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 4, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Chronic Pain and Language: A Topic Modelling Approach to Personal Pain Descriptions

Chronic pain is recognized as a major health problem, with impacts not only at the economic, but also at the social, and individual levels. Being a private and subjective experience, it is impossible to externally and impartially experience, describe, and interpret chronic pain as a purely noxious stimulus that would directly point to a causal agent and facilitate its mitigation, contrary to acute pain, the assessment of which is usually straightforward. Verbal communication is, thus, key to convey relevant information to health professionals that would otherwise not be accessible to external entities, namely, intrinsic qualities about the painful experience and the patient. We propose and discuss a topic modelling approach to recognize patterns in verbal descriptions of chronic pain, and use these patterns to quantify and qualify experiences of pain. Our approaches allow for the extraction of novel insights on chronic pain experiences from the obtained topic models and latent spaces. We argue that our results are clinically relevant for the assessment and management of chronic pain.


Introduction
Chronic pain is a major health problem, with impacts at the individual, social, and economic levels. At the individual level, it dominates a multitude of aspects of the patient's life, in most cases even extending to family and friends. For instance, due to immobility, muscles start to weaken, discouraging further exercise or movement. This can lead to sleep disturbances and a vulnerable immune system, affecting the subject's psychological balance. Adequate chronic pain assessment determines the quality of its management, which has been identified as a key issue in improving the quality of life of these patients (Fink, 2000). Since there is a manifestation of pain that is not directly visible to the outside world that is clinically relevant (Loeser and Melzack, 1999), a verbal interview with a patient is a key moment for pain assessment and management, in which information about the cultural, behavioural, and psychosocial dimensions of the subject in pain are conveyed, intentionally or unintentionally, in the form of expressions. The most common Email addresses: diogo.p.nunes@inesc-id.pt (Diogo A. P. Nunes), jogomes@med.up.pt (Joana Ferreira Gomes), fanineto@med.up.pt (Fani Neto), david.matos@inesc-id.pt (David Martins de Matos) expressions of chronic pain are cries, facial expressions, verbal interjections, descriptions, emotional distress, disability, and other behaviours that come as consequences of these. The expression that is the object of study of the present work is the verbal description of the experience of pain. The description often-times includes valuable information about the bodily distribution of the feeling of pain, temporal patterns of activity, intensity, and others. Additionally, the choice of words may reflect the underlying mechanisms of the causal agent (Wilson et al., 2009), which in turn can be used to redirect the therapeutic processes. Indeed, this forms a specific sub-language which has been studied in previous research, such as the structuring of the Grammar of Pain (Halliday, 1998), and the study of its lexical profile, resulting namely in the McGill Pain Questionnaire (MPQ), which is widely used to characterize pain from a verbal standpoint, in clinical settings (Katz and Melzack, 1999;Sullivan, 1995). However, the identified MPQ pain descriptors represent only a portion of the lexical profile of the language of pain. Some studies have specifically stated that the fixed quality of the MPQ ultimately limits the assessment in terms of stability and predictiveness, concluding that the descriptors should be subordinated to the sociocultural, linguistic background of each patient (Sullivan, 1995). Moreover, the MPQ only accounts for the lexical profile of this sub-language, leaving out of consideration more complex structures, such as syntactic and semantic structures that appear in spontaneous accounts that can convey other informations. Additionally, all of these studies relied on manual methods and human evaluation, rendering it expensive to replicate to other cultures and languages.
The use of NLP techniques has the potential of overcoming all of these limitations. The computational analysis of syntactic and semantic structures of the language of pain may yield correlations between the content of the descriptions and other relevant medical and non-medical aspects of the painful experience, allowing for a systematic and quantifiable way of characterizing pain and disease manifestations on a linguistic level. This, in turn, has the potential to aid health professionals with the clinical assessment and management of these patients. By modelling descriptions of pain in terms of semantic concepts, or topics, it is possible to extract information from text in an unsupervised manner without relying on the explicit analysis of syntactic structures, which is complex and fragile when dealing with spontaneous accounts. This is fundamental in contexts such as transcriptions of natural speech, which, in general, include speech disfluencies not commonly present in written text. This is in line with the data used in this work, which consists of transcribed verbal accounts of experiences of chronic pain, in Portuguese, guided by a fixed interview. The interview script was designed by the authors and health professionals to focus on key cognitive aspects known to influence pain perception and description.
We propose a methodology to take advantage of topic modelling techniques to extract information, in a medically relevant way, from descriptions of chronic pain. To the knowledge of the authors at the time of submission, this is the first proposal for computational analysis of spontaneous verbal descriptions of chronic pain experiences. To summarize, the contributions of our work are: 1. Integrate the analysis of lexical, syntactic, and semantic structures, furthering the reach of the MPQ and MPQ-based clinical analysis; 2. Automatically account for the sociocultural subordination that the MPQ and MPQ-based questionnaires lack (and cannot easily adapt); 3. Present, apply, and discuss the computational method to extract information from spontaneous verbal accounts of chronic pain experiences, the results of which are shown to be clinically relevant; 4. Provide novel ways (based on our methodology) to characterize and categorize chronic pain patients, according to their spontaneous accounts, paving the way for future research to integrate NLP techniques with the clinical assessment and management of these patients.
We start by presenting the data and its collection in a medical environment, in parallel with its intrinsic challenges. Next, we detail and discuss a methodology to overcome such difficulties, followed by the evaluation of the obtained results. Finally, we present approaches to the extraction of insights of chronic pain from the obtained topic models, including the identification of clinically relevant aspects of the experience, and the definition of groups of similar patients.

Background and Related Work
The experience of pain, dependent on its temporal pattern of activity, bodily distribution, and other aspects, is moulded by multi-domain cognitive factors, both individual and sociocultural. Some of these, known to influence pain perception and corresponding suffering, are emotional states, beliefs, expectations, and behaviours (Hansen and Streltzer, 2005;Azevedo et al., 2012). Language has been found to convey part of this information (Melzack and Torgerson, 1971;Wilson et al., 2009). Understanding how the language of pain is used for expressing specific types of pain experiences allows us to build a linguistic model of pain descriptions. Similar descriptions of pain might describe similar characteristics of different experiences of pain. Characterizing these descriptions by their semantic topics allows us to quantify the relations between different experiences in this abstract space. Also, it may be possible to characterize specific types of pain by their associated semantic topics. Even though not directly applied to the language of pain, NLP techniques have seen an increase in health-related applications. An important application is the extraction of mental health features from social media texts, showing that informal language can be used to accurately classify users as having or not mental health problems (Yates et al., 2017;Cohan, Arman and Desmet, Bart and Yates, Andrew and Soldaini, Luca and MacAvaney, Sean and Goharian, Nazli, 2018). Other works focused on extracting entities and relations, such as symptoms, also from social media texts (Foufi et al., 2019;Nzali et al., 2017).
Topic modelling focuses on extracting latent information in a given document from a collection. Plus, topics can be characterized by themselves when they are attributed with "meaning", given the context of a problem. Models, such as LDA (Blei et al., 2003) and NMF (Lee and Seung, 1999), take a vocabulary-based representation of the collection, e.g. Bag-of-Words (BoW) or Term Frequency / Inverse Document Frequency (TFIDF), and extract topics following probabilistic and matrix factorization approaches, respectively. Certain contexts focus on short-text, where the document length shifts from the hundreds of words to the hundreds of characters, such as data from online platforms (e.g. Twitter). In our case, we deal with spontaneous accounts of experiences of pain, which resemble short texts. Extracting topics from short texts presents challenges that the traditional models are not capable of efficiently overcoming, specifically the difficulty in capturing word co-occurrence information, due to noise and sparsity.
Biterm Topic Model (BTM) (Yan et al., 2013) is a probabilistic approach that tackles sparsity in short-text document collections by virtually aggregating the whole collection into a single, long document. Thus, instead of capturing word co-occurrence at the document level, this information is captured at the collection level, so that it describes a generative process of word co-occurrence patterns, denominated biterms, instead of documents.
Latent Feature LDA (LF-LDA) (Nguyen et al., 2015), in contrast, incorporates latent features (semantic representations) learned from external corpora in the LDA model, specifically in the topic-to-word Dirichlet multinomial component. This allows for the external semantic information to model topic-to-word generation when there is lack of information in the documents themselves.
Generalized Pólya Urn Dirichlet Multinomial Mixture (GPU-DMM) (Li et al., 2016) follows a similar approach, but instead of directly incorporating the external semantic information into the generative process, during the estimation step, promotes semantically related words to be assigned to the same topic. The generative process on top of which it is built is given by the Dirichlet Multinomial Mixture (DMM) model (Yin and Wang, 2014) that makes the simplifying assumption that each short-text document talks about one single topic, so that all words in a document are sampled independently from the same topic multinomial distribution, which in turn is sampled from the topic Dirichlet mixture of the collection.
CluWords (Viegas et al., 2019) exploits external semantic information by replacing each term in a document BoW representation by a meta-word, the CluWord, which represents the cluster of syntactically and semantically similar words, as determined by a pre-trained wordembedding model. This enhanced TFIDF/CluWord representation matrix is then submitted to factorization as in the traditional NMF approach. The main limitation of this model lies in the suitability of external resources to the problem domain.
SeaNMF (Shi et al., 2018) overcomes the problems associated with short-text collections by capturing beforehand word and context vectors for the whole vocabulary and explicitly using them instead of the TFIDF representation. This is shown to capture relevant term-context correlation that otherwise would not be fully taken advantage of by the NMF approach. However, the limitation of this model lies in the corpus availability and vocabulary diversity.
LDA-like models under-perform in short-text contexts, when compared with NMF-like models with respect to the Pointwise Mutual Information topic coherence metric (Chen et al., 2019), due to the sparsity and noise of short texts, the instability of stochastic Gibbs sampling when there is insufficient term co-occurrence information, and the fact that NMF can operate in matrix representations of collections which might encode term dis-criminative information, such as TFIDF. For these reasons, we are interested on short-text topic modelling following non-probabilistic approaches, specifically those based on NMF.

Data Acquisition
Our data is the result of a joint collection project with the Faculty of Medicine of Univ. of Porto. It took place at the public hospital of São João, Porto, Portugal (CHUSJ), from Oct. 2019 to Oct. 2020. The dataset contains verbal descriptions of chronic pain experiences, from recorded interviews. The patients are adults (≥ 18 years old), of either sex, diagnosed with osteoarthritis, rheumatoid arthritis, or spondyloarthritis, and with symptoms of chronic pain. The dataset collection project was approved by the Ethics Committee of CHUSJ. Data confidentiality is explicitly protected: all recordings are anonymous, and results are always presented without individual references.

Target information
An interview composed of 7 questions was designed with the aim of obtaining a natural description of the patient's pain experience, in their own words, whilst directing it towards the cognitive topics relevant for pain assessment. The script, validated by multiple health professionals included in the collection process, is as follows:

Acquisition setting, challenges, and pre-processing
Interviews with 94 patients were conducted after the pre-scheduled medical consultation with each patient, using a smartphone as a recording device. Due to the setting of the collection, in a clinical context, there are a number of challenges: some patients showed difficulties in verbally expressing themselves; verbal accounts carry speech disfluencies, such as repetition and correction; and the medical office is not free of random environment noises. The data pre-processing step aims at reducing the impacts of these challenges. Interview recordings were submitted to a semi-automatic diarization process, separating patient audio segments from interview questions. The patient audio segments were manually transcribed, leaving out speech disfluencies. Even though the number of patients is comparable to that of other studies (Carlson et al., 2020;Lascaratou, 2007), it is still an important limitation. To overcome this limitation, we augmented the collection by fragmenting each patient document into the corresponding 7 answers to the interviewer. In this setting, we are considering the answers to each question to be semantically independent. Note that patients are still fully represented by their corresponding 7 documents. As a result, the dataset contains 616 documents, 526 unique tokens of a total of 3872, and an average of 6.3 tokens/document. Fig. 1 shows the document length distribution. We standardize the text through lemmatization with STRING (Mamede et al., 2012), and remove stop-words as defined by the Natural Language Toolkit (NLTK) (Loper and Bird, 2002) for Portuguese, with additional words empirically found not to be semantically meaningful for our task, such as the frequent formal address "senhor doutor" (doctor). Empty documents were removed. Even though speech disfluencies were explicitly removed, texts resulting from spontaneous speech can still be syntactically incoherent and their linguistic analysis cannot be based on the explicit modelling of syntactic structures.

Topic Modelling
The models we apply are LDA, NMF, SeaNMF, and CluWords with pre-trained word-embeddings of both FastText (Mikolov et al., 2017) and BERT (Devlin et al., 2018). LDA and NMF are baselines, and are expected to under-perform in the setting of short-texts. The others are applied in order to understand the gain in the two different types of approaches, in this case, internal and external semantic vocabulary representations, respectively. We define k = 12 as the ideal number of topics to extract from the collection, based on the design of the interview, which includes at least 7 different aspects of the experience, and because it is known that a few patients extended their descriptions beyond these aspects. Previous experiments empirically validated this decision.
Each topic model defines a k-dimensional latent space over the collection of fragmented descriptions. We evaluate the quality of each of these spaces with two metrics, regarding the model interpretability and the spatial document distribution in each topic space.
A topic is a distribution of weights over the vocabulary of the collection. By observing the most weighted words of each topic, we can obtain an idea of the concept(s) it may cover. Considering the various aspects of the experience of pain as dimensions on which to evaluate each patient, we expect the extracted topics to represent, in total or in part, a subset of these dimensions (as given by the interview). A topic that does not share a single word with the remaining topics may define a concrete, modular concept, which allows for an independent evaluation of the projected population on that dimension. Thus, we evaluate topic modularity by determining the number of words shared between topics extracted by any one model (given the top-10 words of each topic). By this metric, we are most interested in the topic model which extracts topics with most unique words. However, the discussion around this metric must take into consideration the actual words of each topic, because, according to it, a modular topic is not guaranteed to be semantically relevant.
How the projected fragments are organized in the latent semantic space reveals intrinsic qualities of that space. Clustering and observing the Silhouette Coefficient (Rousseeuw, 1987) of each sample and cluster are two relevant measures of the spacial distribution of the fragmented descriptions. We look at the Silhouette Coefficient of each sample in each latent space, s = (b − a)/ max(a, b), where a is the mean distance between a sample and all other samples in the same cluster, and b is the mean distance between that sample and all other samples in the next nearest cluster. This provides a measure of the quality of sample distribution across clusters, and how adequate each sample is to the assigned cluster. The mean silhouette score of a given topic model is obtained by averaging the silhouette scores of all samples in the topic space defined by that model.
In our setup (fragmented short-text documents), the typical document projection is composed of a highly weighted dimension and the remaining with infinitesimal values, i.e., only one concept is being discussed. Thus, we expect to obtain the best clustering for a model with the number of clusters close to the number of extracted topics, in this case, c = k = 12.

Results
Tables 1 through 5 show, for each model, the top-10 words of each topic. Starting with the baseline models, we conclude that the top words defining the NMF topics allow for a slightly easier interpretation than those of LDA. Nevertheless, both topic models are still hard to interpret. CluWords (FastText) topics are easier to interpret: some topics seem to relate to concrete concepts, such as pain location, intensity, and treatment. SeaNMF, on the other hand, seems to be overfit to the collection, since the words describing each topic reveal more word co-occurrence rather than conceptual information about the latent space, rendering it very hard to interpret. Fig. 2 shows, for each model, the topic modularity score for each topic, as well as the average modularity per model (Section 4). These results mirror the empirical conclusions taken from topic interpretation: the models which are hardest to interpret have the lowest scores in terms of topic modularity. However, as observed with SeaNMF and noted in the previous section, topic modularity must never be taken into account by itself. Fig. 3 shows the Silhouette Coefficient scores for each sample for each model, for a number of clusters equal to the number of topics, c = k = 12. Baseline models were omitted for ease of discussion. Observing the silhouette score for each sample, rather than the mean value per model, gives us a clear understanding of sample distribution in space and in relation to the assigned clusters, so that we can better assess how well the projected fragments are grouped in the topic space. As expected from previous results and observations, the majority of the projected fragments in the SeaNMF topic space are indistinguishable from one another, and there is a poor distribution of samples per cluster, suggesting that these projection dimensions are not as semantically interesting as those of the CluWords models. For the CluWords models, we observe that there is a better sample distribution per cluster in the CluWords (FastText) topic space, in general with better silhouettes per cluster, albeit with a marginally inferior mean Silhouette Coefficient score.

Discussion
Following the previous observations, we discard the baseline LDA and SeaNMF topic spaces. The remaining models are all based on the same NMF model implementation and parameters, albeit on top of slightly different vocabulary-based representations of the fragment collection, obtained with external FastText and BERT pre-trained word embeddings, respectively for CluWords (FastText) and CluWords (BERT). Results show that even though the vocabulary is very poor in terms of diversity, there are clear gains in using external semantic relations suitable to our domain, as demonstrated by the results of the CluWords models in relation to the baselines. We conclude that the topic space given by CluWords with Fast-Text is the most adequate for our task.

Topic labelling
The topmost weighted words of each topic are the ones most commonly used in similar contexts by the patients, and, thus, relate to some concept being discussed in those contexts. This concept is not necessarily concrete, because it may cover multiple sub-concepts, nor comprehensive, because the words representing the topic may not cover the full meaning of the associated concept, and may be context dependent. Therefore, assigning a label to a topic is both a difficult and biased task, which can introduce errors and limit generalizations. Having this in mind, and because each question in the interview aims at specific aspects of the experience of pain, using only the 10 most weighted words of each topic, we interpret and associate a label reminiscent of the questions in the interview script (Tab. 5), which is in itself a bias. Some topics are more concrete and easier to interpret: treatment, specific body locations, impacts (1, 2), time intervals, evolution, and generic body locations. Others were associated with the concepts in the interview that are expected to prompt the use of such words: activity, actions, causes, intensity, and reflections.
In previous research, chronic pain episodes in clinical texts (reports written by health professionals when assessing patients suffering of chronic pain) have been systematically annotated, finding concepts such as date, pain location, cause, social and emotional effect, and others, to be covered in a high percentage of these clinical reports (100%, 84%, 90%, 78%, respectively), clearly evidencing the importance of these concepts for clinical assessment and management of these patients (Carlson et al., 2020). Because there is a match between relevant clinical concepts (found in the cited previous research) and the topics extracted with our presented methodology (specifically, with CluWords (FastText)), we argue that our results are intrinsically related to the experience of pain itself and are relevant for clinical assessment.

Patient-level topic distribution
In this section we demonstrate a novel way to characterize the experience of chronic pain, according to the patients considered in our dataset. This characterization clearly highlights which dimensions of the experience of pain are more and less important for the average patient, and exactly which allow for a better distinction of patients. Following the previous argument that the dimensions extracted with our methodology are clinically relevant, we argue that this characterization may allow for a new perspective on the clinical assessment of chronic pain patients. If we consider each patient to be fully represented by all answers given to the corresponding interview, uniformly weighted, we obtain topic mixtures at the patient level by aggregating the corresponding fragment projections. Thus, from the original topic projection of dimensions 616 × k, we obtain a patient topic distribution matrix of dimensions 94 × k. We can, thus, discuss topic distributions at the patient level (Fig. 4). Results show that the first three topics (treatment, activity, and specific body locations) are more relevant for patient characterization, since they display the highest variance and mean values, by large margins, with equal prompting. Conversely, results also show that the last three topics (generic body locations, evolution, and intensity) are not relevant for patient discrimination and characterization, since around 75% of the population assigns very low importance to those topics. We also observe that no topic has more than 50% importance to a patient in the collection, which adds to the notion that an experience of pain is rarely uni-dimensional.

Patient-level similarity grouping
In this section we demonstrate a novel way to assess and manage chronic pain patients, by measuring their similarity and aggregating them according to their spontaneous descriptions. This allows us to state that there are different groups of patients (i.e., that differ on how they experience their chronic pain), which patients belong to which group, and leverage other medical and nonmedical information in each group to characterize new patients in relation to the others of the same group. For example, when a new patient is allocated to an existing group, therapeutics and diagnosis of the other patients in that group may be informative to better clinically assess the new patient. Moreover, a similar characterization to that shown in the previous section can be made for each group of patients, so that each individual patient can in addition be characterized by the whole group, potentially revealing new clinical perspectives. Fig. 5 shows a 2D visualization of patient projections and the corresponding assigned clusters and cluster centers. Some clusters are Figure 5: Projected patients on a 2D t-SNE (Maaten and Hinton, 2008) visualization, with color codes referring to the assigned cluster, for a total of 7 clusters. clearly separated from the rest, whilst others have considerable dispersion. Nevertheless, these observations show that there are patients that are similar in terms of how they describe their pain, as given by their topic mixtures, and that there are clear distinct groups. Considering that there is a link between the description and the experience of pain, we can conclude that topic modelling over descriptions of pain may allow for the grouping of patients that share similar aspects of the experience of pain, and that this similarity is quantifiable.

Conclusion and Future Work
In this work, we presented and discussed an approach on how to take advantage of topic modelling techniques to extract interpretable information from spontaneous verbal descriptions of chronic pain. The descriptions were obtained from recorded guided interviews, following a design shaped by the authors and specialized health professionals. We showed the importance of the application of short-text oriented topic models in contrast to traditional approaches, and concluded on the advantage of using externally obtained semantic relations to enhance the quality of the extracted topics. We argued on how the results are semantically relevant and intrinsically related to the experience of pain and to the clinical assessment and management of chronic pain patients. Finally, we demonstrated novel approaches on how to extract insights from the patient level topic distributions, including the identification of the most and least relevant aspects of the experience of pain for our population, and the grouping of similar patients according to how they describe their pain. We also reflected on how these can have an important impact in the clinical assessment and management of chronic pain patients.
To the knowledge of the authors at the time of submission, this is the first proposal to computationally analyse and characterize spontaneous accounts of chronic pain experiences. We believe that the results presented in our work pave the way for future computational analysis of the language of chronic pain, and that this will ultimately provide new insights and techniques for the clinical assessment and management of chronic pain.
Future work, on a first step, should focus on extending the collected data, so that results may be more generalizable. This includes both the inclusion of more patients, as well as the diversity of collection contexts, since the context in which an experience is being described highly influences the language used and the focused topics. With larger corpora, it may then be possible to integrate the proposed analysis with data from other domains. As an example, the proposed similarity system may be used together with diagnosis information, so that the underlying pathology causing chronic pain may be predicted for new entries (patients). Similarly, a therapeutics-suggestion system (to help guide or inform health professionals) may be built on top of the same methodology.

Acknowledgements
Diogo A.P. Nunes is supported by a scholarship granted by INESC-ID Project UNBABEL, with reference BI|2021|133. Additionally, this work was supported by Portuguese national funds through FCT, with reference UIDB/50021/2020.  arm  bad  start  improve  think  walk  bad  day  affect  raise  want  hand  leg  day  movement treatment  wait  knee  back  medication always  can  able to  knee  work  can  walk  disease  evolve  always  hurt  take  can  explain  can  foot  knee  always joint  take  see  little  time  bad  disease  rheumatoid arthritis move  hurt  walk  hold  stiffness  medication origin  back  reach  hurt  day  move  walk  hip  home  affect  hour  time  medication professional professional always  normal  able to  hand  joint  hard  start  body  pass  idea  emotional  height  feel  walk  origin  people  quite  position rest  constant  big  walk  foot  start  get worse  want  foot  improve shoulder  able to  time  provoke  want  trouble  work  neck  continue  people  day  arm  wrist  hip  night  less  able to year day able to walk understand night stop bone