{'fraction_non_alphanumeric': 0.035189788614446146, 'fraction_numerical': 0.011389079882230568, 'mean_word_length': 5.086704582036699, 'pattern_counts': {'":': 1, '<': 0, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 10, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
The Moral Foundations Reddit Corpus

Moral framing and sentiment can affect a variety of online and offline behaviors, including donation, pro-environmental action, political engagement, and even participation in violent protests. Various computational methods in Natural Language Processing (NLP) have been used to detect moral sentiment from textual data, but in order to achieve better performances in such subjective tasks, large sets of hand-annotated training data are needed. Previous corpora annotated for moral sentiment have proven valuable, and have generated new insights both within NLP and across the social sciences, but have been limited to Twitter. To facilitate improving our understanding of the role of moral rhetoric, we present the Moral Foundations Reddit Corpus, a collection of 16,123 Reddit comments that have been curated from 12 distinct subreddits, hand-annotated by at least three trained annotators for 8 categories of moral sentiment (i.e., Care, Proportionality, Equality, Purity, Authority, Loyalty, Thin Morality, Implicit/Explicit Morality) based on the updated Moral Foundations Theory (MFT) framework. We use a range of methodologies to provide baseline moral-sentiment classification results for this new corpus, e.g., cross-domain classification and knowledge transfer.


Introduction
Moral rhetoric and framing play a role in increasing polarization and divisions in our societies (Marietta, 2008;Dehghani et al., 2016;Brady et al., 2020), but also in a wide range of pro-social behaviors that can potentially bring people together (Voelkel et al., 2022;Wolsko, 2017;Kidwell et al., 2013;Moaz, 2020). In order to understand the relationship between hate, division, compassion, and unity in the digital age, we need to understand * Equal contribution the dynamics of moral language online. In particular, capturing and investigating the moral sentiment of text can allow for the study of how individuals' and groups' expressed moral sentiment relate to various downstream online and offline behaviors.
Moral sentiment assessment and classification are subjective tasks, and when done automatically using Natural Language Processing (NLP) techniques, this subjectivity results in the need for large and diverse, both in terms of topics and coders, sets of annotations. The Moral Foundations Twitter Corpus (MFTC; Hoover et al., 2020), a collection of 35,108 tweets that have been curated from seven distinct domains of discourse and hand annotated for 10 categories of moral sentiment (care, harm, fairness, inequality, loyalty, betrayal, authority, subversion, purity, and degradation) based on the Moral Foundations Theory (Haidt, 2012;Graham et al., 2011), was released a few years ago. This corpus has been used to design novel methods for moral sentiment classification (Asprino et al., 2022;Lan and Paraboni, 2022;Burton, 2022;Araque et al., 2020;Wang and Liu, 2021;Wu and Huang, 2022), used in models to investigate the impacts of moral framing in other domains (e.g., misinformation and polarization, Mutlu et al., 2020;Ruch et al., 2022), and has been applied to train models that produce morally salient text (e.g., arguments and jokes, Alshomary et al., 2022;Yamane et al., 2021). However, as useful as MFTC is, its training dataset is limited to the social-media platform Twitter.
Different online social media platforms have different linguistic and social structural environments that may result in variations in moral language and behavior (Curiskis et al., 2020). Beyond differences in social structure, different platforms have varying character limits (e.g. 280 characters on Twitter compared to 10k-40k on Reddit) which alters the language usage of users (Boot arXiv:2208.05545v2 [cs.CL] 18 Aug 2022 et al., 2019) and therefore may contribute to differences in use and effectiveness of moral rhetoric (Candia et al., 2022). Additionally, different platforms have different policies with respect to the levels of user anonymity and sensitive content moderation which may additionally influence the domains of morality discussed given the potential judgements from others. Research has argued that higher levels of anonymity reduces the feeling of responsibility and alters moral behavior online (Simfors and Rudling, 2020). Lastly, modern NLP methods are known to require massive training data for producing sufficiently accurate, generalizable, and robust models. It has empirically been shown that diverse sets of training data, from different platforms and on different topics, can help improve the classification results by allowing the models to obtain generalized domain knowledge (see Kennedy et al., 2022a), rather than surface knowledge restricted to a particular platform and a small set of topics.
As mentioned previously, the MFTC relied on the Moral Foundation Theory's framework which is a pluralistic perspective of moral cognition and identifies multiple dimensions of moral values that have evolved to facilitate individual well-being, coalitional unity, and cooperation with strangers (Haidt, 2012;Graham et al., 2013). The original version of the theory identified five separate but interrelated categories of moral concerns: Care/Harm, Fairness/Cheating, Loyalty/Betrayal, Authority/Subversion, and Purity/Degradation. Recently, a revision to the Moral Foundations Theory (Atari et al., 2022a) split Fairness into distinct and new foundations of Equality and Proportionality, while retaining the other four existing foundations of Care, Loyalty, Authority, and Purity. This split aims to capture the distinct moral concerns of fairness in procedure (proportionality) and equality of outcome (equality). In order to better understand the different nuances that will result from this theoretical change, we need to have an updated annotated corpus that complies with the latest theoretical revisions.
Together, the above reasons call for a corpus from a different platform, focused on a diverse set of topics and annotated by a diverse group of annotators. Here we address this need by introducing the Moral Foundations Reddit Corpus (MFRC), a collection of 16,123 Reddit.com comments annotated for 8 categories of moral sentiment and cu-rated from 12 morally-relevant subreddits.
Reddit is a public social media platform with approximately 52 million daily users who post content in over 138,000 active "subreddits" which are user-created and user-moderated communities about different subjects (Upvoted Staff, 2021). Shared content and comments in subreddits are "voted" on by users which is used to decide the visibility of the post. Activity on Reddit has been the center of a number of prominent cultural moments including coordinated attempts to challenge short-sellers of GameStop stock (Roose, 2021) or attempts to identify the Boston-city bombing terrorists (Starbird et al., 2014).
We focused our corpus compilation effort on Reddit for a number of reasons. First, in comparison to Twitter, Reddit shares many of the same research friendly features (e.g., responsiveness to current events, public posts, and available APIs) (Proferes et al., 2021), but is organized into what are called subreddits. Different subreddits have distinct topics and consistent communities (Datta and Adar, 2019;Soliman et al., 2019) with varying cultures and norms (Chandrasekharan et al., 2018;Fiesler, 2019). In relation to morality, these distinct communities and norms have led researchers to use Reddit to investigate moral conflicts across groups (e.g., Kumar et al., 2018), a phenomena that is harder to investigate on Twitter (which does not have organized groups) or Facebook (where many groups are private). Second, Reddit provides more anonymity than many other social media platforms, potentially enabling users to more freely speak their minds and express their opinions (e.g., Triggs et al., 2021;De Choudhury and De, 2014;Simfors and Rudling, 2020). Third, in addition to general differences in language usage (Boot et al., 2019), the lack of restriction on the length of posts on Reddit may be particularly beneficial for training models. Fourth, we believe that Reddit has played a distinct role in contemporary politics. For example, the r/TheDonald and r/incels subreddits have been linked to political extremism (Gaudette et al., 2021) and mass shootings (Helm et al., 2022).
In order to understand the validity and relative performance of different text classification methods for identifying the moral concerns manifested in the MFRC, we report baseline results for a number of NLP approaches: Distributed Dictionary Representations (DDR; Garten et al., 2018), Bidi-rectional Encoder Representations from Transformers (BERT;Devlin et al., 2018), andMulti-Label (Tsoumakas andKatakis, 2007) BERT models. These baselines can both be used as evidence about the relative performance of various NLP models on the task of moral concern detection and also inform future methodological work including to help calibrate new models of moral sentiment classification.
Finally, in order to facilitate research into annotator response patterns and bias, as recommended by Prabhakaran et al. (2021), we provide psychological and demographic metadata of our annotators. The background and biases of human annotators have been shown to impact their annotations (Hovy and Prabhumoye, 2021;Davani et al., 2022Davani et al., , 2021Bolukbasi et al., 2016) with particularly damaging effects that amplify pre-existing biases (Mujtaba and Mahapatra, 2019;Zhao et al., 2017). Annotators' biases may be particularly relevant in domains characterized by high subjectivity, such as moral values (Garten et al., 2019a,b;Hoover et al., 2020). While, for example, an annotator's political ideology might not have a substantial influence on how they annotate "positive" and "negative" sentiment in a corpus of hotel reviews, it seems likely that their ideology could substantially influence how they annotate expressions of justice and respect in a politically relevant corpora.
In the rest of the paper, we provide a detailed description of the corpus, our annotation procedures, a set of baseline classification results using a range of methods, and a cross-corpus comparison with MFTC.

Corpus Overview
As noted above, the MFRC consists of 16,123 Reddit comments drawn from 12 different subreddits. These subreddits were chosen based on the following criteria: First, we focused on subreddits that we expected to contain a wide range of moral concerns. Second, the chosen subreddits had to be active and have sufficient data. Third, we aimed to have a non-US based political subrreddit with focus on current events that could be of use for different research communities. Accordingly, our corpus consists of 12 subreddits organized into three buckets; US politics, French politics, and Everyday moral life. The US politics bucket contains comments from 3 subreddits from the dates 1/1/2020 -1/31/2021; r/politics which captures political moral language generally, r/conservative which covers moral rhetoric of the right, and r/antiwork which covers different, but still political moral language from the left. The everyday moral life bucket is a collection of topics related to various aspects of everyday life, collected for their non-political moral judgement and moral emotions which includes comments from the 4 subreddits of r/nostalgia, r/AmitheAsshole, r/confession, and r/relationshipadvice between the dates of 1/1/2020 -1/31/2021. The third bucket on French politics and contains comments from the subreddits of r/europe, r/worldnews, r/neoliberal, r/geopolitics, and r/Conservative that had the relevant keywords related to the presidential race including 'Macron', 'Le Pen', 'France', 'French', and 'Hollande' (see below for the full set of keywords used) from the dates of 01/01/2017 to 06/30/2017 and had at least 10 likes/comments in order to control for sufficient engagement.

General Sampling Procedure
In assembling the MFRC, we sampled Reddit posts from a larger set of each subreddit. Our initial filtering criteria selected posts of sufficient length (at least 10 tokens), along with de-selecting any posts that were automatically marked by reddit as a bot, with the text "I am a bot" appended to the end of the post. The French politics bucket was also filtered for comments that mentioned at least one of the French presidential candidates ('macron', 'le pen', 'hollande', 'dupontaignan', 'hamon', 'arthaud', 'poutou', 'cheminade', 'lassalle', 'melenchon', 'asselineau', 'fillon') in which most hits were for frontrunners Macron, Le pen, Hollande, and Fillon. For the US Politics and Everyday Politics buckets, comments were selected that had a comment score of at least 10.
In this filtered set, the proportion of comments that contained moral sentiment proved too low to conduct fully randomized sampling in a way that would result in a sufficient amount of moral examples. While these subreddits were chosen specifically for their potential moral salience, research has shown that use of moral language in some do-mains are rare (Atari et al., 2022b). To address this issue, a semi-supervised method was used to up-sample from moralized posts (Kennedy et al., 2022b). Specifically, we first used word embeddings and a list of moral foundations seed words to compute a moral loading score (DDR; Garten et al., 2018) for every comment. Next, for each moral concern, we computed the 95% percentile scores to mark the highly moral comments and set bin size. Finally, in order to have a diverse range of moral posts, we compiled comments in a manner that each subreddit bucket consisted of 1/2 comments with high moral loading (> 95%), and 1/2 comments with less high moral loading (≤ 95%).
This filtering and sampling procedure yielded approximately 6,000 comments for US Politics, 6,000 for Everyday Politics, and 8,000 comments for French politics. However, since vice, virtue, and multiple foundations regularly co-occur in an individual comment, duplicates occurred and were subsequently removed, resulting in a smaller final sample size for each bucket (US Politics: 4,821; French Politics: 6,489; Everyday Morality: 4,813; Total: 16,123).

Annotation
Every post in the MFRC has been labeled by at least three trained annotators from a set of five (see Table 1 for the distribution of annotators for the corpus) for 8 categories of moral sentiment as outlined in the new version of our annotation manual, the Moral Foundations Coding Guide-2 (see Appendix).

Moral sentiment
Moral sentiment labels are drawn from the recently revised typology of Moral Foundations Theory (Atari et al., 2022a), which proposes a six-factor taxonomy of morality. In this model, each factor includes both virtues, or prescriptive moral concerns, and vices, prohibitive moral concerns. The proposed moral foundations are (see Atari et al., 2022a): Care/Harm: Intuitions about avoiding emotional and physical damage or harm to another individual. It underlies virtues of kindness, gentleness, and nurturing, and vices of meanness, violence, and abuse.
Equality/Inequality: Intuitions about egalitarian treatment and equal outcome for all individuals and groups. It underlies virtues of social justice and equality, and vices of discrimination and prejudice.
Proportionality/Disproportionately: Intuitions about individuals getting rewarded in proportion to their merit (i.e., effort, talent, or input). It underlies virtues of meritocracy, productiveness, and deservingness, and vices of corruption and nepotism.
Loyalty/Betrayal: Intuitions about cooperating with in-groups and competing with out-groups. It underlies virtues of patriotism and self-sacrifice for the group, and vices of abandonment, cheating, and treason.
Authority/Subversion: Intuitions about deference toward legitimate authorities and high-status individuals. It underlies virtues of leadership and respect for tradition, and vices of disorderliness and resenting hierarchy.
Purity/Degradation: Intuitions about avoiding bodily and spiritual contamination and degradation. It underlies virtues of sanctity, nobility, and cleanliness and vices of grossness, impurity, and sinfulness.
It should be noted, that unlike the MFTC, in the MFRC we did not code for Fairness concerns. Rather, following the latest theoretical developments in the field (Atari et al., 2022a), we coded for separate foundations of Proportionality and Equality.
In addition to these six foundations, annotators were trained to look for an additional construct: Thin Morality -a moral judgment or concern which is voiced without clearly referring to one of the six moral domains (see Atari et al., 2022b). This brings the total categories of moral sentiment to 7. Annotators also had a formal category for Implicit/Explicit Morality -whether the moral sentiment in the comment was expressed explicitly or implicitly. Lastly, the annotators were asked to report their overall level of confidence in their annotation as not confident, somewhat confident, or very confident.

Annotators
We started with a larger pool of 27 annotators, all undergraduate research assistants who completed two months of training sessions to develop expertlevel familiarity with MFT. Training consisted of lectures, discussions, readings, and practice annotations with inter-annotation agreement analysis. In early annotation stages, annotator disagreement was addressed through discussion and, if necessary, certain labels were modified. However, given the subjective nature of the task, in many cases, it is difficult to make the determination of whether or not a document expresses moral sentiment or which category of moral sentiment it expresses. While it is necessary to have consistent annotator training, a focus on maximizing annotator agreement risks artificially inflating agreement at the cost of suppressing the natural variability of moral sentiment (Hoover et al., 2020). Accordingly, our annotators were trained to both strive for consistency, while also encouraged to avoid stereotypes that may increase agreement with other annotators but would lead them to ignore their own beliefs. Out of these original set of annotators, we selected the top five performing annotators, based on both inter-coder reliability assessments and the commitment of the annotators to the project, to become primary annotators and complete the rest of the annotations in our corpus.
Our trained annotators were independently assigned to label each comment from a subset of comments sampled from a corpus associated with one of the 12 subreddits (see Table 2). The annotations were performed on Prodigy. 2 Each post was assigned a label indicating the absence or presence of the six foundations, thin morality, explicit/implicit expression, and the confidence level of the annotator (see Figure 3 in the Appendix).

Annotator Metadata
We have also collected responses to a range of psychological and demographic measures from each of our annotators. While keeping their identities anonymous, we provide measures of each an-notator's gender (Man, Women, Non-binary/third gender, Other, Prefer not to say), sexual orientation, age, household income, first language, political ideology along a liberal-conservative scale, religious affiliation, and Moral Foundation Questionnaire-2 (Atari et al., 2022a). Basic analyses demonstrate that our annotators' political ideology and morality skews liberal while family income skews wealthier than the average American. Based on the recommendations of Prabhakaran et al. (2021), in order to increase utility and transparency of this corpus, we include these measures and encourage research into their potential impact on annotations, and the subsequent biases in the machine learning models.

Annotation Results
The annotation results can be seen in Table 2. It should be noted again that each post was annotated for multiple labels by multiple annotators, and the frequencies reported in Table 2 are calculated based on annotators' majority vote (i.e. posts receiving at least 50% agreement for that label). For example, if a particular post is annotated as 'proportionality' by at least 50% of the annotators who coded that post, then the majority vote on 'proportionality' for that post is positive. We acknowledge the uneven distribution of labels across the various subreddits, in addition to the low baserates of annotated posts for some the moral concerns, especially for Purity, Loyalty and Proportionality concerns.
The interannotator agreement results, using Fleiss's (Fleiss, 1971) kappa and prevalence-and bias-adjusted Fleiss's kappa (PABAK; Sim and Wright, 2005) for multiple annotators, are displayed in Figure 1. Fleiss's kappa is generally viewed as the gold standard measure for investigating agreement across many annotators, and it represents the degree of agreement beyond what is expected by chance. This measure though, is heavily influenced by the prevalence of positive cases. Given the subjective nature of our task, and the fact that positive cases are not prevalent given the often rarity of moral rhetoric (Atari et al., 2022b), we use PABAK which adjusts kappa for prevalence and bias. As expected, given the low base rate of moral language across the various subrredits (i.e. low positive cases), most reported kappa's are low. However, once adjusted for the issue of prevalence, we see medium to high agree- ments across the subreddits.

Baseline Classification Models
In addition to compiling the MFRC corpus, we experiment with different models and provide classification baselines for predicting the annotators' majority vote of moral sentiment categories. Our goal here is to simply establish baselines that fu-ture work can build on. It should be noted once more that each post in MFRC was coded by multiple annotators for 8 different categories of moral sentiment discussed under the Section 3. This is a multi-label classification task; i.e., not only the categories of moral sentiment are not independent of one another, but understanding variance in one domain should theoretically inform about other related moral domains. Here though, we provide both single-label (treating labels as independent) and multi-label classification results.
DDR-SVM As the simplest approach, we used a Support Vector Machines (SVM; Joachims, 1998) to predict each category of moral sentiment based on feature representations generated by the DDR method. Given sets of words related to each of the six moral foundations, first, each post is represented as an embedding. Then, the cosine similarity between the average word embedding representation of each Reddit post and the set of words representing each target moral foundation is calculated. The list of the seed words used for DDR can be found in the Appendix Table 20.
BERT classifier In second approach, we finetuned BERT sequence classifier for each individual label separately for 5 epochs, with a learning rate of 2e-5, and batch size of 8.
Multi-label BERT classifier Lastly, we finetuned Multi-Label BERT sequence classifier to predict all moral categories at the same time for 10 epochs, with a learning rate of 2e-5, and a batch size of 8.
We used the Huggingface Transformers package (Wolf et al., 2019) for BERT tokenizer and se-quence classification. Due to the sparsity of the data (i.e., the number of moral posts being much smaller than the number of non-moral posts), we used the following weighted loss function where the sample i's weight in loss value of label l is proportional to the inverse of the label frequency: The baseline metrics we report are the F 1 , precision, and recall, which are calculated across stratified 10-fold cross-validation. For the multi-label BERT, we stratified the data based on the moral sentiment label (the union of all the moral labels). We used 10% of the training data for validation, and we chose the best performing model based on binary F 1 for single-label classification and F 1 macro for multi-label BERT. For the SVM models, we used grid search cross-validation from the Scikit-learn package (Pedregosa et al., 2011).

Results
The results of the baseline models are provided in Tables 3 to 10. We provide both the average of each of the three metrics, along with the standard deviations, across the 10-folds. We ran each of the baseline models once for all the corpus, and once separately for each of the three buckets. As expected, BERT models outperform DDR-SVM models in terms of F 1 and Precision. However, DDR-SVM models achieve higher Recall in almost every category of moral sentiment. To our surprise, multi-label BERT achieved lower performances than the vanilla BERT model.

Cross-Corpus Classification
In the previous section, we provided performance of different baseline models' on the MFRC corpus. In this section, we evaluate preliminary results on the transferability of models between MFTC and MFRC. Our hope is that future work can build on such cross-domain tasks in order to extract more generalized knowledge about moral rhetoric independent of the source and topic of the post.
In order to provide a fair comparison between MFRC and MFTC, we trained BERT models with the same hyperparameter discussed in the previous section. Additionally, we downsampled the MFTC dataset to have the same number of samples for each label as MFRC. The results are presented in Table 11. In general, models trained and tested on the MFTC have better classification performance than the results we provided in the last section for the MFRC.
As mentioned before, in the MFRC, we rely on the updated taxonomy of Moral Foundations in which the Fairness foundation is split into Proportionality and Equality. This makes the crosscorpus training for Fairness, Proportionality and Equality difficult. We hope to update the MFTC in the coming months and use the more nuanced Proportionality and Equality labels in that corpus as well. For now though, for predicting Fairness labels in the MFTC, the union of Proportionality and Equality labels are calculated based on the MFRC trained models, and the output of the union is then compared to the Fairness category in the MFTC. Similarly, to evaluate the MFTC Fairness models on the MFRC, both Proportionality and Equality are assigned the Fairness label predicted from the MFTC.
Cross-corpus results are presented in Tables 12 and 13. These preliminary results are indeed encouraging in that we demonstrate transferbaility between the two corpora in predicting out-ofdomain distributions. Previous research training classifiers on the MFTC, and testing on Reddit (e.g., Atari et al., 2021) have shown similar levels of accuracies for cross-domain classification. Any cross-corpus investigation should take into account the different time periods in which these two corpora were compiled. This difference can potentially impact the topics, the sentiments expressed about the topics, and the type of justification and reasoning used for the expressed sentiments. We believe though that more advanced methods in knowledge capture and representation could use the two corpora together to further achieve more generalized and better performing models.

Discussion
Moral rhetoric and framing has been shown to be predictive of various important pro-social and anti-social behaviors. Several NLP methods have recently been proposed for capturing and categorizing moral sentiment based on textual data (for a review see Atari and Dehghani, 2021). However, this is a subjective sentiment analysis task for which training data plays a vital role. To facilitate further research in this domain, here we introduced the MFRC, a collection of 16,123 Reddit         comments annotated for 8 categories of moral sentiment, and provided a number of baseline results for different NLP models trained to predict moral sentiment.
The MFTC was introduces in 2020, and so far this corpus has already facilitated multiple lines of research in both NLP and the social sciences. We believe Reddit's distinct linguistic and social structure, along with MFRC's methodological and theoretical updates, allow for potential new research that can both improve and expand the applications of MFTC. Specifically, the increased character limits on Reddit compared to Twitter is important for the more naturalistic expressions of moral rhetoric and its potential impact on the performance of classification models. While social media often provides large amount of data needed for training NLP models, with respect to sentiment analysis of moral language, the paucity of moral rhetoric in some domains (Atari et al., 2022b) makes it difficult to gather sufficient amounts of training data (Hoover et al., 2020). Given Reddit's longer posts, models trained on the MFRC may perform better in out of domain tasks, especially in longer documents (e.g., articles or speeches compared to tweets). Further, the distinct subreddit communities allow for the study of group linguistic dynamics. For example, shifts in moral language over time associated with a hashtag on Twitter may show the evolving general public opinion on a topic, while shifts in moral language in a particular subreddit may reflect the shift in opinion of a particular group. This is important in respect to accounting for the down-  stream group behaviors associated with moral language use such as the voting behaviors. Moreover, the increased anonymity on Reddit can facilitate research into the gap between identity-linked and publicly-expressed moral concerns, such as on Facebook or Linkedin, and anonymous expressions of moral values, as posted on Reddit.
As discussed previously, another important feature of the MFRC is that it is based on the newly updated version of the Moral Foundations Theory (Atari et al., 2022a) which breaks the Fairness concern into the distinct moral concerns of Proportionality and Equality. The MFRC can be used to further investigate these nuances of fairness across topics such as income inequality and excessive wealth.
Similar to the MFTC, the MFRC is supplemented with detailed meta-deta on the annotators of the corpus. We hope that by providing demographics and several key psychological measurements of our annotators, MFRC can facilitate future research into how annotators background characteristics impact their annotations.
In conclusion, we hope that the MFRC, along with this report, in addition to MFTC and the other previously released corpora in this domain (e.g., Kennedy et al., 2022b), can aid researchers by providing much needed data and open new lines of research both in NLP and in the social sciences.
In an age where political movements, grass roots activism, and plans for insurrections seem to take place in online environments, it is vital that we can better understand the moral dynamics of these online conversations. The intent of this project has been to further facilitate research into these timely topics.

Data Disclaimer
We acknowledge that the compiled dataset contains biases and is not representative of diverse moral concerns present in world populations. Potential biases in the data include: biases specific to English-speaking countries and the English language, biases inherent to Reddit.com and its user base, biases in the researchers' criteria for corpus curation as well as the underlying MFT itself, bias in the assessment of moral labels, and the fact that annotators were all undergraduate research assistants at a private academic institute. All of these factors, among others, likely influenced the annotations, as well as the performance of machine learning models trained on the corpus. Anyone using this corpus should be aware of these limitations and should acknowledge and/or try to mitigate them to the extent possible.

Acknowledgements
This research was sponsored by NSF CAREER BCS-1846531 and DARPA HR001121C0165 to MD. Each of the five foundations in MFT is conceptualized as having solved different adaptive problems in humans' evolutionary past (Haidt, 2012). The Care foundation accounts for our nurturing of the young and caring for the infirm. The Fairness foundation accounts for the development of human cooperation, justice, and reciprocity. Loyalty is concerned with coalition-building with ingroup members, Authority is concerned with respecting high-status individuals in social hierarchies, and Purity is about physical cleanliness and spiritual sacredness of objects, humans, and groups.

References
One strength of MFT, as formulated by Graham et al., is its openness to new foundations, with the idea that plurality is the most important concept for understanding human morality, and that the specific set of five foundations originally proposed in MFT are just one proposed set of foundations. Recently, Atari et al. (2022a) proposed breaking Fairness into two more narrowly-defined foundations, "Equality" and "Proportionality". Equality describes people's concern with similar outcomes or status (e.g., a violation of Equality is systematic racial inequality, the state of individuals of different races having different access to resources and opportunities). Proportionality describes people's desire for balance between actions and responses. Proportionality concerns are typically centered around the ideas of meritocracy and deservingness. For example, cheaters should be punished, hard workers should be rewarded, and slackers should be excluded relative to the extent of their contribution.
When people express their moral attitudes, emotions, and concerns about people, actions, events, concepts, and ideas, they employ diverse rhetorical strategies (Keen, 2015). Often, these strategies rely on words that are explicitly normative, such as "right," "wrong," "good," or "bad" (which we call "thin morality"; see Atari et al., 2022b); however, in many cases, people communicate a moral attitude by communicating the relevance of a moral domain (e.g., Care). For example, "I can't believe that happened. It's so harmful!", "People should be compassionate," or "This decision hurts so many people!" position the discussed entity or topic as either aligned or misaligned with "good" morality, by assuming the virtue or desirability of care centered actions, people, or things.
The six moral foundations (i.e., Care, Equality, Proportionality, Loyalty, Authority, and Purity) have a natural mapping to language, which can be instantiated by identifying words which are used by speakers to communicate their attitudes with respect to each moral foundation. In the above examples, speakers' attentiveness to the Care foundation is apparent from their usage of the words "harmful," "compassionate," and "hurt." Moral foundations in language were first studied in this way by Graham et al. (2009), which provided the Moral Foundations Dictionary (MFD) (for a detailed discussion of language analysis in moral psychology, see Atari and Dehghani, 2021). The MFD considers each moral foundation, including the "vice" and "virtue" poles of each foundation, as a collection of related words, the usage of which indicates a concern with the given foundation. In early work, Graham et al. (2009) used the MFD to measure differences in moral values sentiment between conservative and liberal sermons. More recently, researchers have shown that moral value annotation based on the MFT taxonomy can fruitfully be applied to a range of applications and domains (Dehghani et al., 2016;Sagi and Dehghani, 2014) While the above findings, facilitated by the MFD, have shed light on the nature of morality in "the wild" by analyzing observational text data, there are reasons to question the use of the dictionary approach for measuring moral phenomena in text. There is an implicit assumption that the frequency of certain explicit moral words indicates an underlying concern with the corresponding moral domain. For example, MFD findings articulate implicitly that using moral words implies a higher concern with morality at the individual level. Recently, however, Kennedy et al.
(2021) tested this assumption using responses to the Moral Foundations Questionnaire (MFQ; Graham et al., 2011) and participants' Facebook status updates. Among other findings, this work established that the existence and size of the relationship between explicitly moral language (i.e., the MFD) and individuals' moral concerns was inconsistent across foundations. Specifically, Care, Fairness, and Purity language had a positive correlation with the corresponding moral domain in language (e.g., high Care concerns implied the usage of Care words), while Authority and Loyalty did not. Moreover, other techniques such as topic modeling, which represent all words (and not just explicitly moral ones) predicted significantly more variance in individual-level moral concerns than did methods based on explicitly moral language.
As far as the present coding guide is concerned, these findings imply that the domain of language which is related to moral concerns is far wider than purely explicit moral language. In terms of the annotation of moral phenomena in text, we will take the view that moral concerns can commonly be communicated without the presence of explicitly moral words, e.g., "No matter what, it's [my team] forever! 10-0 or 0-10, nothing changes for me" is an expression of loyalty to a sports team without explicitly using words like "loyalty."

A.1.3 Thin Morality
Not all moral language falls within the scope of the six domains indicated by MFT. In fact, philosophers have denoted two types of morally evaluative language: Evaluative terms and concepts are often divided into "thin" and "thick". We don't evaluate actions and persons merely as good or bad, or right or wrong, but also as kind, courageous, tactful, selfish, boorish, and cruel. The latter are examples of thick concepts . . . [which] stand in contrast to those we typically express when we use thin terms such as right, bad, permissable, and ought (emphasis in original; Väyrynen, 2016) In our above section on moral concerns in language, we have been discussing thick morality. In our annotation, we will attempt to comprehensively cover all types of moral language by also annotating thin morality.

A.1.4 Annotator Uncertainty and Authorial Inferences
There is unavoidable ambiguity that affects text annotation, which is key to understand when conducting annotation-based studies of moral concerns in language. Our approach to ambiguity in this guide is to use background context to inform annotations, but also to report the level of uncertainty for a given annotation. The major source of ambiguity is caused by the difficulty of inferring the moral content intended by an author. For example, a social media message might simply state that the author thinks "Everything that is going on with abortion these days is reprehensible." In this case, it is clear that this is likely a morally relevant statement, but it is less clear what foundation this statement is relevant to. If we knew that the author was concerned with civil rights, we might assume that the author is concerned about violations of women's reproductive rights (i.e., an instance of Equality). In contrast, if we knew that the author was a conservative Christian, we might assume that the author was expressing an anti-abortion sentiment, perhaps associated with Purity.
These ambiguities present considerable challenges for human annotators who must strike an acceptable balance between exploiting often weak signals of moral sentiment while also avoiding unfounded speculation about author's intent. In this guide, we recommend that annotators tend to focus on objective sources of confidence for resolving ambiguities of intent. However, since so much in the domain of moral language is not objective (i.e., relying on author assumptions), in this guide we propose the usage of a "Confidence" label, which is designed to allow annotators to assign a label based on an inference about intent, but to also indicate a lower confidence in the label.

A.2 Instructions for Annotators: Annotating moral concerns in language
Annotating moral concerns in language involves determining whether a given text's author is communicating a moral attitude, emotion, judgment, or moral issues toward particular persons, groups, questions or problems, or event. In this section, we provide instructions for annotators for the identification of moral concerns in text. We emphasize the six domains of moral language based on MFT; detail the target and vice/virtue com-ponents of moral concerns in text; detail instructions for annotators to assign a confidence rating to each annotation; and describe the particular approaches annotators ought to use for different language types (e.g., social media versus transcribed speech).

A.2.1 Annotation Task
For moral concern annotation in text, annotators should complete, in order, the four subtasks outlined in Figure 2. Below, we will explain each subtask.  (1) and (2) categories are not present; and (4) assess confidence in label.
Label Thin Morality ...contain evaluative language, such as calling someone or something good, bad, right or wrong?
(3) If you selected no "Yes" labels for (1) and no "Thin Morality": (4) How confident are you in your annotation? Select Very Confident, Somewhat Confident, or Not Confident

Label Not Confident
Annotating Moral Domains We first annotate text by categorizing text into non-mutually exclusive "domains" of moral concerns, which are the six foundations of MFT.
In Table 14, we list the six foundations, giving their name, a definition, and an example item from the recently developed MFQ-2 (Atari et al., 2022a). Items were presented to participants with the prompt, "Please indicate how well each statement describes you or your opinions." Even with clarity as to the conceptual domains described by each of the six foundations, it is not straight-forward to map these categories to language. Below, we describe three distinctions, or components, of a voiced moral concern: (1) the explicit/implicit distinction, (2) the con- I believe that compassion for those who are suffering is one of the most crucial virtues.

Equality
Intuitions about egalitarian treatment and equal outcome for all individuals and groups. It underlies virtues of social justice and equality.
The world would be a better place if everyone made the same amount of money Proportionality Intuitions about individuals getting rewarded in proportion to their merit (e.g., effort, talent, or input). It underlies virtues of meritocracy, productiveness, and deservingness.
The effort a worker puts into a job ought to be reflected in the size of a raise they receive

Loyalty
Intuitions about cooperating with ingroups and competing with outgroups. It underlies virtues of patriotism and self-sacrifice for the group.
I believe the strength of a sports team comes from the loyalty of its members to each other

Authority
Intuitions about deference toward legitimate authorities and high-status individuals. It underlies virtues of leadership and respect for tradition.

I think obedience to parents is an important virtue
Purity Intuitions about avoiding bodily and spiritual contamination and degradation. It underlies virtues of sanctity, nobility, and cleanliness.
It underlies the widespread idea that the body is a temple that can be desecrated by immoral activities and contaminants (an idea not unique to religious traditions) crete versus abstract target distinction, and (3) the vice/virtue (positive/negative) distinction.
Explicit and implicit expressions of moral concerns Each of these six concerns can be invoked in language in ways that can be either implicit or explicit. Explicit invocations will use words that map clearly to the domain in question; a sample of such words are given in Table 15. The presence of these words in a document/sentence/utterance hints at the presence of a moral expression, but does not necessarily confirm it. An actual invocation of a given moral domain will use these words in a particular way. Differentiating what we might call "moral" uses of these words from "non-moral" uses is similar to the challenge of "word sense disambiguation" in Natural Language Processing. Word sense disambiguation acknowledges that words can have multiple uses or "senses" depending on the context in which they were used. For example, the word "fairly" can be used in a Care sense, e.g., "Humanity is we treat every person fairly, even when we're threatened," but can also be used in a non-Care sense, e.g., "There's fairly universal protocol on how to treat anyone who makes dumb decisions".
Other times, moral concerns can be invoked implicitly (i.e., without using explicitly moral language). This category of moral expression is harder to specify a priori due to its myriad forms; here, annotators should rely on their understanding of the concepts underlying each moral domain, as denoted in the flowchart in Figure 2 and in Table 14. For example, it is possible to express a concern about Equality without explicitly naming the Equality domain: "AT USER it's a shame Skin color and beliefs fuel hatred" communicates a concern regarding the fair treatment of people based on skin color or beliefs. Similarly, it is possible to invoke the Loyalty domain without using explicit loyalty words: "Rep voter [suppression] efforts in Florida a disgrace to Americans my Dad who fought in WWI for freedom & democracy" describes the virtue of loyalty to American veterans without using the word "loyalty" (etc.). The takeaway from these examples is that moral domains can be invoked by understanding the meaning of the text in question, and is not limited to the presence or non-presence of explicitly moral words.
In summary, there are two types of language which signal the presence of a moral concern: 1. Explicitly moral words used in a moral way 2. Any language used to express the speaker's moral concerns, attitudes, emotions, behavior, or beliefs about some persons, actions, objects, events, ideas, etc.
Next, we detail two sets of distinctions for moral expressions that can help annotators to identify different types of moral language.
Concrete versus abstract targets of moral expressions People often use moral values when they are expressing a judgment about someone or something -i.e., the object. The object of a moral judgment can be either someone (e.g., a person or a social group) or something (e.g., a behavior, an event, an abstract concept, or even a physical object). The object of a moral judgment can be concrete (e.g., judging a specific behavior or person) or abstract (e.g., judging a general value or opinion). When annotating moral foundations in natural language, we are not, per se, interested in the object of a moral judgments. However, identifying the object of a moral judgment can sometimes help clarify whether the moral judgment in question is related to one of the moral foundations.
Vice and virtue expressions The types of moral judgments individuals make about people or things can be either positive or negative. For example, a person might praise someone for engaging in moral behavior or condemn someone for engaging in immoral behavior. That is, a moral judgment entails a positive or negative evaluation of the object of the moral judgment. In broad terms, an expression of virtue communicates that "good should happen" while an expression of vice communicates that "bad should not happen"-what is "good" and "bad" depends, of course, on which moral concern is being evoked.
An evaluation is positive when it calls for moral actions, praises people for moral behavior, or lauds a moral value or opinion. An evaluation is negative when it decries immoral actions, criticizes people for immoral behavior, or condemns an immoral value or opinion. While we are not, per se, interested in distinguishing between positive and negative moral judgements, this distinction can sometimes help clarify what moral foundation is being invoked.

A.2.2 Annotating Thin Morality
For our purposes, thin morality is a moral judgment or concern which is voiced without clearly referring to one of the six moral domains. For example, the Tweet "Why does [he] reply to profane and disrespectful tweets from rude constituents he's a good guy" (emphasis added) makes a statement about the goodness of an individual but does not describe the individual's goodness on account of a particular moral domain.
We note that Thin morality is in fact mutually exclusive with thick morality, and thus if a document is, for example, annotated with Loyalty, it cannot also be labeled as Thin morality. Unlike our definition of thick morality (i.e., the presence of one of the six MFT domains), the presence of thin morality is most often marked by the presence of words (e.g., right, wrong, better, worse, good, bad)

A.2.3 Annotator Confidence
After completing annotation of moral domains and Thin morality, and regardless of whether or not Non-Moral was selected, annotators should select one of three confidence labels: Very Confident, Somewhat Confident, and Not Confident. These are fully defined in Table 16.

A.3 Language Domains and Appropriate Annotation Strategies
The goal of this coding guide is to be domainagnostic. That is, rather than a guide for specifically Twitter (e.g., Hoover et al., 2020) or Facebook data (e.g., Atari et al., 2022b), we aim to provide a framework that is flexible enough to guide annotations for any textual domain, includ- No moral expression in the text, but possibility the speaker could be implying a moral concern Somewhat Confident A moral expression in the text, but possibility that the speaker is using sarcasm or similar One or more moral domains are clearly present, but at least one is vague or uncertain No moral label, but with more context it might be possible to establish that the author did intend to communicate a moral concern Not Confident One or more moral labels, but with more context it might be possible to establish that the author did not intend to communicate anything moral Two or more moral domains are equally present, but there is no way to resolve either confidently ing social-media posts, comments on online posts or articles, published text, literary text, historical pieces, and transcribed speech.
Here, we note particular strategies annotators should take for each language domain.
Social media (Twitter, Facebook, Instagram, etc.) Social media text is typically short form, containing incomplete sentences, abbreviations, hashtags and 'at'-mentions, and hyperlinks. For our annotation, we ask that annotators ignore atmentions and hyperlinks (including media) and to infer as much as possible from the available context. In some cases, this might include references to current events; for example, the Tweet "No Social Security number should mean no claim to any benefits or credits. #takeastand" references the process in the United States whereby a social security number grants access to certain governmentprovided services. In other cases, abbreviations (e.g., BLM) can be used in ways that add meaningful information. For example, the sentence "We have endured too much!" might be labeled as Loyalty; however, with the inclusion of the BLM hashtag -"We have endured too much! #BLM", this might additionally be considered Care or Equality, given that the Black Lives Matter movement is focused on harms and systematic inequalities directed toward Black persons. Annotators are asked to look up unfamiliar abbreviations that occur frequently, or words that seem to have a unique use in a particular online platform or group; however, if the abbreviation is not obvious or frequent, it can be ignored. Hashtags, particularly if they are themselves abbreviations, can be used to resolve context. However, they should not be used as the sole reason for labeling a document as a given moral domain. For example, the text "Having some fries with my drink #Equality" contains a relevant hashtag that does not help to resolve ambiguity, and thus should be ignored. Lastly, hashtags that are used fluidly in a sentence (e.g., "#Dreamers play a vital role in our communities") can be treated as normal words.
In addition to general social media considerations, each platform has specific components that inform annotation strategies. For Twitter, posts are embedded in a networked context with sharing ("Retweeting") and conversational components. Currently, we do not support the ability to view conversational context when annotating a Tweet, though this might change in the future. Retweets, marked with "RT" at the beginning of the Tweet, and Quote Tweets, marked by a preceding remark followed by the main Tweet in quotes, should be viewed as endorsing the message contained in the original tweet, and annotated accordingly. Lastly, most of the considerations that apply to Twitter apply to other short-form social media, such as Instagram.
For Facebook, posts can be longer (i.e., multisentence or multi-paragraph), requiring more time than Tweets and other short form text. However, Facebook posts are typically less ambiguous than Tweets, as they contain more context. The moral concerns voiced in a Facebook posts will likely be contained in one or two select sentences. Ad-ditionally, it is more likely for a Facebook post to contain multiple moral concerns than it shorter media like Twitter.
Online comments (Reddit) The distinguishing characteristic of online comment language is its referencing of original posts. For example, a comment on a Reddit post in the forum "Am I the Asshole" will be making judgments or comments about a post in which the original author explained a personal story, asking anonymous ethical judges to pronounce judgment on the individuals in the story. For our purposes, we will not have annotators read original posts in large part due to the length of original posts. Instead of relying on this context, we will ask annotators to label comments using only the language contained in the given comment. This has limitations with regard to resolving ambiguities, and thus annotators should take care to report annotator confidence when additional context would be needed to label an ambiguous comment.
Transcribed speech Spoken language is altogether different from written language, due to the difference between spontaneous conversation and the premeditated nature of written text. Transcriptions of speech capture all the artifacts of speech, including "er" and "ah" sounds, short sentences, and incomplete sentences. Also, the types of moral concern voiced in spoken language tends to be more concrete than abstract. The following examples illustrate the types of language in (transcribed) spoken text: "That's good. I'm happy you're taking care of you [sic] mom" (Care); "I will never leave your side" (Loyalty); "I'm your dad! You need to respect me!" (Authority); "John is a good man" (Thin Morality).
Published text/articles Lastly, published text or articles, whether sampled at the sentence, paragraph, or document level, require annotators to read carefully and to consider as much external context as possible. Similarly to Facebook posts, annotators should attempt to identify sentences or sequences of sentences that contain a give moral concern. Additionally, given the nature of published text and articles, more is known about the speaker, or at least the speaker's objectives, which might be to persuade readers about a certain point or to describe a story or event. This context can be used to resolve ambiguities in text. Figure 3: Here is the Prodigy annotation tool interface with a sample comment on top, followed by the moral sentiment categories and confidence levels options, with navigation to the next sample on the bottom.

A.5 Examples
A.5.1 Examples of each Label from the MFTC Telling someone (object) to be a true patriot (virtue) No to illegal immigrants-they need to follow the process, obey the law @realDonaldTrump #EndDACA

Authority
Expresses that people should obey (virtue) the US president (object) @realDonaldTrump "If you love me, obey my commandments." -Jesus John 14:15

Authority
Telling some (virtue) to obey God's commandments (object) Glad to see a reformation going on to restore sanctity of free speech Purity Praising (virtue) a restoration of the sanctity of free speech (object) It's absolutely disgusting how every retailer exploits a serious storm situation by peddling their crap

Purity
Expression of disgust (vice) regarding exploiting a storm situation to sell wares (object)