{'fraction_non_alphanumeric': 0.0298661506208676, 'fraction_numerical': 0.011127237542331882, 'mean_word_length': 4.909281494187154, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Misinformation Detection using Persuasive Writing Strategies

The spread of misinformation is a prominent problem in today's society, and many researchers in academia and industry are trying to combat it. Due to the vast amount of misinformation that is created every day, it is unrealistic to leave this task to human fact-checkers. Data scientists and researchers have been working on automated misinformation detection for years, and it is still a challenging problem today. The goal of our research is to add a new level to automated misinformation detection; classifying segments of text with persuasive writing techniques in order to produce interpretable reasoning for why an article can be marked as misinformation. To accomplish this, we present a novel annotation scheme containing many common persuasive writing tactics, along with a dataset with human annotations accordingly. For this task, we make use of a RoBERTa model for text classification, due to its high performance in NLP. We develop several language model-based baselines and present the results of our persuasive strategy label predictions as well as the improvements these intermediate labels make in detecting misinformation and producing interpretable results.


Introduction
The idea of misinformation has always existed, but within the past few years, the amount of misinformation, specifically concerning information related to health and the medical industry, has skyrocketed (Suarez-Lledo and Alvarez-Galvez, 2021), and it has taken a toll on society. People's trust in news organizations has fallen drastically, and that has resulted in a less informed populace (Islam et al., 2020). Many news resources purposefully make false news articles in order to spread misinformation and increase hostility between political groups. A bulk of prior online misinformation research focuses on the political context (Lazer et al., 2018). However, health-related misinformation has grown exponentially during the COVID-19 pandemic and these types of misinformation have a significant impact on individuals and society (Loomba et al., 2021). There are simply huge amounts of data for human fact-checkers, more than what they can go through manually, so professionals have turned to machine learning to solve this problem. Automated misinformation identification has been a problem that researchers have been trying to solve for years, and it is still a prominent challenge. Moreover, we recognize that it is not very helpful in practice to have a machine learning model that merely identifies misinformation even with high accuracy. For people to trust a machine's verdict in misinformation, the models must also produce interpretable reasoning as to why an article is real or fake. In this paper, we present a novel annotation scheme containing an exhaustive hierarchy of persuasive writing strategies systematically identified from online misinformation (Peng et al., 2022). An example of the type of labels in the proposed hierarchy is shown in Table 1. We use these strategies so that once trained, a model can identify which parts of an article cause it to be labeled as untrustworthy. If a user learns what persuasive strategies are used in the source of information, they will be more likely to have a deeper analysis of them and be less likely to fall for misinformation. Our contributions

Parent
Child Sub Establishing Surface Credibility Scientific Legitimacy Markers Jargon

(Health) Misinformation
Several terms have been used to describe inaccurate or false information, including misinformation, disinformation, fake news, conspiracy theory, and rumor. Some researchers consider misinformation as an umbrella term to describe all false and inaccurate information lacking scientific evidence, no matter whether such false information is created or disseminated intentionally or unintentionally (Swire-Thompson and Lazer, 2019). Misinformation is not a new concept. Systematic research on rumors in mass media and deception in interpersonal communication dates back more than 70 years ago (Allport and Lepkin, 1945;Knapp, 1944). What makes the current misinformation stand out is its ability to transcend temporal and geographical barriers as well as the enhanced capability in search, archive, and access (Fernández-Luque and Bau, 2015). In particular, for social media, misinformation can spread more quickly through connected clusters of online social networks (Vosoughi et al., 2018). More importantly, the unique features of social media facilitate the creation and maintenance of information silos, filtered bubbles, or echo chambers, rendering contemporary misinformation, especially concerning (Shu et al., 2017).
In this study, we focus on health misinformation, which is defined as health-related information disseminated on the Internet that is false, inaccurate, misleading, biased, or incomplete, which is contrary to the consensus of the scientific community based on the best available evidence (Peng et al., 2022). Before the COVID-19 pandemic, health misinformation has already attracted the attention of researchers due to the insurgence of childhood vaccination misinformation on social media . The burgeoning health misinformation during the COVID-19 pandemic (Kouzy et al., 2020;Cuan-Baltazar et al., 2020) and its associated negative impacts (e.g., vaccination hesitancy (Loomba et al., 2021;Roozenbeek et al., 2020)) makes it all more important to better identify and counter health misinformation.

Combating Misinformation from Machine Learning Perspective
Due to the immense amount of news that is produced every day, it is unrealistic to expect human fact-checkers to label all. Many have turned to machine learning models to aid the process of claim verification and misinformation detection. As per (Rani et al., 2022), the number of papers published relating to rumor and fake news detection has grown exponentially since 2006. Put simply, misinformation is the presentation of any purposefully falsified information. Within misinformation, there are multiple categories, including Large Scale Hoaxes (Yuliani et al., 2019;Della Vedova et al., 2018), Satire (Burfoot and Baldwin, 2009;De Sarkar et al., 2018), andPropaganda (Martino et al., 2020;Khanday et al., 2021). Our work focuses on misinformation relating to health and medical fields.

Datasets
There are many datasets that were constructed to solve the problem of misinformation detection. These datasets tend to vary greatly in attributes and content. All datasets contain claims and labels from fact-checking websites such as Politifact , Snopes (Popat et al., 2016), or both (Vo and Lee, 2019). In recent years, more datasets have been created that crawl from a variety of different fact-checking websites in order to increase generalizability (Shahi and Nandini, 2020). Besides claims and labels, datasets often include evidence pages and metadata as well, from sources such as Google (Augenstein et al., 2019) and Wikipedia (Aly et al., 2021). Some recent datasets have included Temporal and Spatial Information so that models can learn from more than just textual data (Shu et al., 2018). Of all of these datasets, none include the text from the original article that the claim originates from. Our dataset uses claims from the MultiFC dataset (Augenstein et al., 2019), along with the original article for more accurate context.

Various Problem Settings
Automated misinformation detection is a problem that can be approached in a variety of different ways. The first part of this problem is labeling.  (Kotonya and Toni, 2020a). The problem of misinformation detection can be defined as classifying a claim, along with its relevant metadata and evidence, as the label that factcheckers assigned to it. Our work seeks to classify entire news articles with these labels, as opposed to singular claims.

Related Techniques
Misinformation detection is a complicated problem to solve. There is a multitude of Natural  vlin et al., 2018). Due to its modified pre-training process, it achieves state-of-the-art results on many natural language processing benchmarks.

Integrating A Persuasive Communication Perspective to Misinformation Detection
The previous section reviewed the current Computational efforts to detect and flag misinformation. However, the deliverable of these efforts is usually a simple label indicating whether the information is true or false or evidence supporting or denying claims. The simple label may not work as expected and only has a limited effect in diminishing the influence of misinformation (Margolin et al., 2018). One step forward from the simple fact-checking label is to provide an explainable justification for why certain claims are misleading (Kotonya and Toni, 2020b;Atanasova et al., 2020). The research in communications has long discovered that pro-viding justification as well as understanding the underlying techniques used in claims to influence the information receiver will be critical to help individuals cope with persuasive attempts (Friestad and Wright, 1994). Previous studies (Eisend and Tarrahi, 2022; Guess et al., 2020) have demonstrated that when individuals become aware of the persuasive tactics or have adequate media or information literacy to understand these persuasive attempts, they become less susceptible to persuasion attempts such as political propaganda and advertising. Such media literacy interventions coupled with fact-checking were found to be more effective than each component used along (Hameleers, 2022). To our understanding, virtually no research is available to provide explanations of persuasive tactics for misinformation detection or mitigation. The current study attempts to extract/automatically annotate persuasive features in misinformation which can both serve as intermediate features for classifying misinformation and as auxiliary information for future media literacy training tools to assist individuals in better misinformation detection.
The persuasive features implemented in the current study were based on a review that systematically screened 1700 research articles related to online health misinformation to identify 12 thematic groups of persuasive strategies from 58 eligible articles (Peng et al., 2022). These 12 thematic groups of persuasive strategies commonly seen in online health misinformation are: fabricating narrative with details, using anecdotes and personal experience as evidence, distrusting government or pharmaceutical companies, politicizing health issues, highlighting uncertainty and risk, inappropriate use of scientific evidence, rhetorical tricks, biased reasoning to make a conclusion, emotional appeals, distinctive linguistic features, and establishing legitimacy. Some of the themes also include subgroups. For example, for the theme of politicizing health issues, there are subgroups such as the trope of freedom and choice, the rhetoric of ingroup vs. outgroup, citing political figures or political arguments, and the use of religion and ideology. Although some similar approaches are available, such as bias detection based on political ideology (Baly et al., 2020), propaganda detection based on linguistic features (Rashkin et al., 2017) or using multiple categories of features (Da San Martino et al., 2019Dimitrov et al., 2021), persuasive strategies based on a particular theory such as Aristotle's rhetoric theory (Chen et al., 2021) or Moral Foundation Theory (Lin et al., 2018), the persuasive strategies identified and annotated in our study have a competitive advantage because they were based on a systematic and comprehensive review of current literature to provide a full-scale framework.

Dataset Construction
For this project, we have chosen to use a subset of MultiFC (Augenstein et al., 2019), consisting of claims relating to the health and medical field. Articles labeled as health, medical, or food were added to the subset. The original source was then retrieved manually by searching the claim on Google. A total of 607 articles were retrieved. However, only true, mostly true, mostly false, or false articles were annotated. Unproven, satirical, and non-health or non-medical related articles were excluded from the annotation process. Additionally, if the original source had been removed online or was in a non-textual format, it was excluded from the annotation process as well, resulting in a total of 249 articles. They were then imported into We-bAnno, where they were annotated with persuasive writing techniques.
There are three levels of labeling in our annotation scheme. These levels can be described as the parent tag level, the child tag level, and the subtag level. The parent tag level is the overarching group of strategies, corresponding to the 12 thematic groups of persuasive strategies identified in (Peng et al., 2022). The child tag level is the subgroup of strategies that fall into a parent tag, and are more specific, e,g., (Politicizing health issues-Religion/Ideology, Emotional Appeals-Fear). The third level is the sub-tag level, which is used when even more specificity is needed, e.g, (Establishing legitimacy-Surface Credibility Markers-Medical/Scientific jargon). Additional subgroup child tags and the sub-tag level were added to the original framework in (Peng et al., 2022). Because the child tag and sub-tag levels are not exhaustive representations, it is possible for a text sequence to be annotated with a parent tag without being annotated with any of its' respective child tags. In the same way, a text sequence can be annotated with a child tag without being annotated with any of its' respective sub-tags. However, if a text sequence is annotated with a child tag, it must be annotated with its respective parent tag, and if a text sequence is annotated with a sub-tag, it must also be annotated with its respective child tag. This rule is illustrated in Table 2.

Labels
Validity Emotional Appeals Valid Emotional Appeals, Valid Emotional Appeals-Anger Emotional Appeals-Anger Invalid A master's student in health communication was trained to annotate the persuasive strategies in the 249 articles by a communication professor. During the human coder training phase, both coded 30 articles. Once discrepancies were resolved (the codebook addresses the issues of discrepancies), the master's student independently annotated the rest of the articles. An intercoder reliability test using a random sample of 10% of the annotated articles (n = 25) revealed that Kappa was 0.851 and percent agreement was 86.1%.
The support values of each persuasive strategy can be found in the Appendix (Table 8). 1

Problem Setting
We define a number of misinformation detection and characterization tasks in this section. Task1: Persuasive Strategy Classification: Given an article A that can be represented as a sequence of n sentences s 1 , s 2 , ...s n , the task is to classify each sentence s i in A with a predefined set of persuasive strategy labels ps = {l 1 , l 2 , ...l m } . The labels l i follow a given hierarchical structure. Each sentence s i can have multiple labels. Thus this problem can be seen as a hierarchical multilabel classification problem per sentence while the context of the article can be used as a part of the input information for the sentence classification. Task 2: Misinformation Detection: Given an article 'A', return a value that corresponds to the trustworthiness of the article T w. T w in its simplest case could be a binary label indicating the truth of the document, that is, T w = {T rue, F alse}. However, detecting the veracity of textual data is often more complex than this, so label sets typically include values including {M ixed, U nverif iable, Outdated, ...}. In this paper, we use the binary labeling scheme. Task 3: Using Persuasive Strategies for Misinformation Detection: This is a setting where the two above tasks can be combined. Since the persuasive strategies potentially can help both the detection and characterization of a news document, we define this third combined task. This task can be set with various configurations including 1) pipeline: predict ps labels, and use the output as the input of the detection. 2) Joint setting: given the document predicts both ps and Tw labels, jointly.

Models
Here, we describe the models designed to conduct experiments for the aforementioned three tasks and evaluate the results. The Transformer-based language models such as BERT excel at many Natural Language Processing tasks and can serve as strong baselines. Specifically, RoBERTa has impressive performance in many NLP tasks. Due to this, we use a pre-trained RoBERTa-Base model as the basis for our experiments. We use a RoBERTa-base Tokenizer for pre-processing the raw input text. BaseModel: This is a RoBERTa-base model used for the detection task (Task1) by adding a classification layer fine-tuned with subsets of the Multi-FC training dataset. The input to this model is a claim, along with evidence text that was retrieved from MultiFCs' evidence pages. This evidence is concatenated to the claim by separating tokens in between, then tokenized. The output is a vector of probabilities with respect to the 9 unique classes that the POMT domain contains, {P T rue , P M ostlyT rue , P Half T rue , P M ostlyF alse , P F alse , P P antsOnF ire , P Half F lip , P F ullF lop , P N oF lip }, where P l is the probability of the input article being classified as label l. HealthModel: This model is designed to test the performance we can get with our health data subset. The fact that our subset contains instances from multiple domains means that some instances can have labels with nearly identical meanings (Ex. Truth!, True). This similarity causes great difficulty in the training process. To mitigate this, we introduce label normalization in later experiments. The input to this model is a tokenized article. The output is a vector of probabilities with respect to the subset classes, {P mostlytrue , P truth! , P true , P in−the−green , P mostlytruth! , P mostlyf alse , P f iction! , P f alse }, where P l is the probability of the input article being classified as label l. HealthNormModel: A model trained on the same data subset, but with the introduction of label normalization. Our label normalization strategy can be found in table 10. This model uses true and false labels. The input to this model is a tokenized article. The output is a vector of probabilities with respect to the binary labeling scheme, {P T rue , P F alse }. HealthWeightedModel: A model trained with the same label normalization, but with the introduction of weighted loss to combat data imbalance. The input is a tokenized article, and the output is the same binary layer used in HealthNormModel. Level1Model: A binary classification model is designed to detect if the text follows a persuasive writing strategy or not. The input to this model is a tokenized text sequence extracted from an article. The text is labeled at the sentence level. This text sequence includes a focus sentence and can include nearby sentences as context if specified. This format is used for all level models. It's output can be described as {P noP ersuasive , P persuasive }. Level2Model: A model designed to detect which parent tags (Table 8) a text sequence should be annotated with. A sequence can be annotated with no tags or multiple tags. The input to this model is the standard tokenized text sequence, including context if specified. It's output can be described as {P P 1 , P P 2 , ..., P P n }, where number of labels is n=12. Level3Model: A model designed to detect which child tags a text sequence should be annotated with. A sequence can be annotated with no tags or multiple tags. The input to this model is the standard tokenized text sequence, including context if specified. The output can be described as {P C1 , P C2 , ..., P Cn }, where n=30. Level4Model: A model designed to detect which persuasive strategy sub-tags according to Table 9 are conveyed in a given text. A sequence can be annotated with no tags or multiple tags. The input to this model is the standard tokenized text sequence, including context if specified. The output can be described as {P S1 , P S2 , ..., P Sn }, where n=9. DetectionModel: A RoBERTa model adapted to make use of article text, along with persuasive strategies, and predict the veracity of the article. The input to this model is an article, along with a string of token-separated persuasive strategies that apply to the text. The output is a binary vector of probabilities, {P T rue , P F alse }.

Experimental Results
Firstly, we want to establish that our base model architecture is competitive to the state-of-the-art and can outperform strong learners on the task of misinformation detection. We accomplish this by training and testing a baseline on a subset of the MultiFC dataset and comparing our results to the results of their best model. We then train the baseline model on our data subset in order to further investigate our experimental questions and hypotheses. Moreover, we discuss the process and results of our automatic strategy classification and misinformation detection. Our experimental results and analysis are to answer the following research questions: Q1. Is RoBERTa baseline competitive with stateof-the-art models on the MultiFC dataset for the task of misinformation detection?
Q2. How well the RoBERTa baseline can predict veracity of articles from text alone, with their original labels?
Q3. How does the introduction of label normalization help the baseline to learn from the healthrelated data subset?
Q4. Can basic techniques such as weighted loss help in dealing with the class imbalance of our dataset?
Q5. How well do the strong baselines predict the various types of persuasive writing strategies in the text sentences?
Q6. How does the inclusion of ground-truth persuasive strategies affect a model's ability to classify the misinformation in the articles?
Q7. How does the inclusion of our models' predicted persuasive strategies affect a model's ability to classify the misinformation in the articles?

MultiFC Baseline Comparison
To answer Question 1, we ran BaseModel on a subset of MultiFC, called the POMT domain, and compared it with the results of their best-performing model reported in (Augenstein et al., 2019). For this task, we crawled the snippets provided in the MultiFC dataset to retrieve evidence for or against claims. This evidence was concatenated to the claim, with separating tokens between the different evidence pages. We feed this format of string into RoBERTa, with a maximum sequence length of 512. The POMT domain consists of 9 labels, some of which contain close to no instances. This results in low accuracy and F1 Scores, as shown in Table 3  As portrayed in the resulting table, our RoBERTa model has lower accuracy and a higher F1 score than MultiFC's model. This tells us that MultiFC's model is more biased towards classes with a high number of instances, while RoBERTa has a more balanced representation. Though the results show the problem is challenging, the RoBERTa model is competitive compared to the previous SOTA for this task.

Health Domain Baseline Results
To obtain results on our health data subset, we performed a set of experiments designed to answer questions 2-4. For these experiments, we used an epoch number of 6, a learning rate of 2e-5, a batch size of 4, and a maximum sequence length of 512. To answer question 1, we trained and tested Health-Model on the subset, using the original article text and the original labels. Because our subset contains instances from multiple domains, it also contains a large number of similar labels. There are a total of 12 unique labels in the dataset. This large number of labels resulted in very low accuracy and F1, showing that the model could not learn well. The results are shown in table 4.
To answer question 3, we ran HealthNormModel on our subset. For our models to learn, a form of label normalization was required. We grouped all labels into {True, False}. Some articles were pulled from the MultiFC test set, which does not contain labels. These articles were labeled as None.
The results of the model after normalization was applied are in Table 4. Now the model can learn the difference between the normalized classes more effectively.
To answer question 4, we ran HealthWeighted-Model on our subset next. We discovered that MultiFC was imbalanced, it contained more false claims than true. This issue of imbalance classes holds for our selected subset too. After normalization, our subset contained~150 False instances and ~60 True instances. Due to this, we decided to test how weighting our Cross Entropy loss would affect performance. These results are also in Table 4. The accuracy was on par with the unweighted model, but the Macro F1 saw nearly a 10% increase. This shows that our model is capable of more unbiased classification.

Detection of Persuasive Strategies
The next question to answer is Q5 on the prediction of persuasive strategies. To accomplish this, some data preprocessing was required. For this preprocessing, note the annotated documents are originally in the form of json files. The article text and persuasive labels are extracted from the json files. Articles are then split into sentences through the use of an NLTK punkt tokenizer. The task can be performed at word or paragraph levels, however, we found the sentence-level assignment of the labels to be more sensible and effective. Sentences could be marked with no annotations, or they could be marked with multiple parents, children, and subtags in the hierarchy of persuasive strategy labels. We tested sentence-wise classification with 3 levels of context: None, Low and High. The Low context includes 2 additional sentences; One to the left of the focus sentence and one to the right. High context includes 4 additional sentences; Two to the left of the focus sentence and two to the right. We tested RoBERTa with 3 levels of classification. Level1Model tested a binary classification of sentences marked with any persuasive strategy versus those not conveying any. The task of Level2Model was, given a text sequence, classify which parent tags it falls under. The task of Level3Model was, given a text sequence, classify which child tags it falls under. The task of Level4Model was, given a text sequence, classify which sub-tags it falls under.
Here, we show some examples of the target persuasive strategy labels taken from the health domain dataset: Ex 1: "I started researching the issue for myself, and was, quite frankly, horrified at what I discovered." ⇒["Emotional Appeals", "Fear"] Ex 2: "Obviously, if you've already developed a sensitivity or allergy to wheat, you must avoid it. Period." ⇒["Distinct Linguistic Features", " Linguistic Intensifier"] Ex 3: "Monsanto, the manufacturer of Roundup claims that application to plants at over 30% kernel moisture result in roundup uptake by the plant into the kernels." ⇒[ ] We use the machine learning platform PyTorch for all experiments. We use the pandas library for storing all datasets. We use the NLTK library for sentence-level tokenization. The hyperparameters chosen for our models are shown in Table  5. The batch size and learning rate are standard values used in text classification. For maximum sequence length, the value is increased as context increases to account for the added text. For the number of epochs, the value is increased as the levels increase to account for the large number of labels. 50% dropout is standard for a pre-trained RoBERTa model.

Detection Using Persuasive Strategies
To answer Q6 on the influence of persuasive strategy labels, we trained and tested our Detection-Model on the articles and the ground truth persuasive strategies they were annotated with. The goal was to test if including persuasive strategies as inputs helps the model to detect the misinformation more accurately. We compared the results of this experiment with the results of HealthWeighted-Model, our best-performing model. First, articles were inputted into the models for levels 1-4. In our configuration, the model outputs were made into a token-separated string. The token length of this string was calculated, then the article text was summarized using the Gensim summarizer so that the article text and all persuasive strategies could fit into the RoBERTa input layer. The results of this experiment are shown in Table 7. In this Table, DM refers to DetectionModel. From these results, it is clear that in our configuration, the inclusion of ground truth persuasive strategies improves the accuracy of misinformation detection by a substantial amount. The information gained from the annotations is valuable to the classification task.
Finally, to answer Q7, we trained and tested our DetectionModel on the articles and our predicted persuasive strategies. This process was identical to the previous experiment, but instead of the target annotations, it includes the predictions of our 4 level models. The predictions were made using high-context sequences, which yielded our most accurate results. The results of this experiment are shown in Table 7. We observe that the results are similar to the results of HealthWeightedModel with no included strategies. This shows the integration of inaccurate predicted persuasive strategy labels hurts the final model's performance in misinformation detection.
Based on these two last experiments, we can see that as the accuracy in detecting persuasive writing strategies in text improves, our ability to detect misinformation is improved. In addition, the byproduct of this effort is the explainability of the misinformation decisions made by the model.

Conclusion
In this paper, we emphasize the importance of producing interpretable results for misinformation detection models to promote transparency and public trust. We have introduced a novel annotation scheme, containing a multitude of persuasive writing strategies systematically identified by communication scientists. We created a dataset annotated with these strategies by human annotators. We design deep learning architectures built on pre-trained language models and train them with the annotated data to automate the characterization of the employed strategies in writing. We discuss the challenges of persuasive strategy classification, including a large number of strategy labels and a limited amount of data. Finally, we discuss the process of misinformation detection using persuasive strategies. Specifically, we show that a highly accurate model for detecting the strategies will be beneficial for both the detection and explainability of misinformation decisions. This dataset and the new proposed challenge can be used by the natural language processing community. We hope that our Full Annotation Table  Parent Child