{'fraction_non_alphanumeric': 0.029907633142190023, 'fraction_numerical': 0.009374034268447618, 'mean_word_length': 4.90032414910859, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Aggregating User-Centric and Post-Centric Sentiments from Social Media for Topical Stance Prediction

Conventional opinion polls were usually conducted via questionnaires or phone interviews, which are time-consuming and error-prone. With the advances in social networking platforms, it’s easier for the general public to express their opinions on popular topics. Given the huge amount of user opinions, it would be useful if we can automatically collect and aggregate the overall topical stance for a specific topic. In this paper, we propose to predict topical stances from social media by concept expansion, sentiment classification, and stance aggregation based on word embeddings. For concept expansion of a given topic, related posts are collected from social media and clustered by word embeddings. Then, major keywords are extracted by word segmentation and named entity recognition methods. For sentiment classification and aggregation, machine learning methods are used to train sentiment lexicon with word embeddings. Then, the sentiment scores from user-centric and post-centric views are aggregated as the total stance on the topic. In the experiments, we evaluated the performance of our proposed approach using social media data from online forums. The experimental results for 2016 Taiwan Presidential Election showed that our proposed method can effectively expand keywords and aggregate topical stances from the public for accurate prediction of election results. The best performance is 0.52% in terms of mean absolute error (MAE). Further investigation is needed to evaluate the performance of the proposed method in larger scales.


Introduction
People usually express their opinions in social occasions with friends and to the public. To know what the general public think about a specific topic, it usually takes much human efforts in designing questionnaires, collecting feedbacks and analyzing them. It's time-consuming and error prone. Depending on the participation of people, there could be not too many effective responses. With the advances of social networking platforms, it's very easy to post articles and reply with comments. For example, Twitter, Facebook, and Instagram are among the most popular social networking sites with different functions. This facilitates users to make online discussions in an immediate way. Given huge amount of social opinions, it would be useful if we can automatically collect and aggregate the general stances from them.
There are some challenges to the problem. Firstly, given the very diverse contents in social media, it would be difficult to obtain the most relevant contents from huge amount of data. Secondly, people might express their opinions in different ways. It would be difficult to extract what they really think about specific topics from very short texts in social media.
Content in a short text is usually limited in scope. Without explaining the ideas and referencing related documents, we might only obtain fragmented terms or named entities just from the sole content of a single post. It might even contain emotional feelings or noises that cannot help us clarify the main idea.
On the other hand, users have different types of activities in addition to posting. For example, most social networking platforms provide mechanisms for making friends, following people or topics that you are interested in, and expressing agreement or disagreement, replying, or commenting on others' posts. These social relations, both explicit and implicit, provide useful clues for understanding what people really think, in addition to what they explicitly mention in post contents. This makes it possible to analyze user opinions by extracting social relations and discovering the major concepts.
In this paper, we propose to aggregate usercentric and post-centric stances for topical stance prediction. Firstly, given the simple topical keyword, we expand the concepts by clustering topic-related posts and comments by their word embeddings, and extract major keywords from each group using word segmentation and named entity recognition methods. Then, given word embeddings, sentiment classification is done by machine learning methods including Naïve Bayes (NB) and Extreme Learning Machines (ELMs) (Huang, 2015). Finally, we aggregate topical stances using both post-centric and user-centric sentiments. In post-centric views, the more positive feedbacks a post gets, the more positive it is regarding the topic. In user-centric views, the more positive comments a user gives, the more positive the user is regarding the topic. By aggregating both post-centric and user-centric sentiments, we are able to analyze the influences of user posts from broader aspects.
In the experiments, we collected data from the most popular online discussion forum in Taiwan called PTT. For sentiment analysis on short texts, we found inconsistent sentiment between user ratings and post contents. After adjustment, ELMs are more stable in sentiment classification performance than Naïve Bayes classifiers. By aggregating stances on three groups of candidates in the 2016 Taiwan Presidential Election to predict the election result, the best performance can be obtained for ELMs with the MAE of 0.52%. This shows the potential of our proposed approach in stance prediction. Further investigation is needed for different types of social media in larger scales.

Related Work
Sentiment classification is one of the major techniques for social media analysis and opinion mining. Documents are classified by overall sentiment instead of topic. For example, Pang et al. (2002) first utilized machine learning techniques in learning classifiers for positive and negative movie reviews. They found features as important factors in social media sentiment classification. Conventional bag-of-words models do not distinguish between word orders. Word n-gram models such as bigrams simply consider consecutive words as a unit for representing documents. It's only limited in the local context of words. Nowadays, word embedding models such as Word2Vec (Mikolov et al., 2013) or GloVe (Pennington et al., 2014) have been used as a more suitable representation of documents, especially for short texts in social media. They utilize neural networks to learn the semantics of words in different contexts. Furthermore, different deep learning methods have been used to automatically learn the features in sentiment classification. For example, Recurrent Neural Networks (RNNs) and their variants such as Long Short-Term Memory (LSTM) and Gated Recurrent Units (GRUs) are often used to capture long-term dependency in sequential data. They have been successfully applied in sentiment classification of tweets (Wang et al., 2018). Convolutional Neural Networks (CNNs) were originally used in image recognition. With suitable representation of word embeddings in documents, CNNs were also found effective in sentiment classification of tweets (Severyn and Moschitti, 2015).
Based on sentiment classification of a single review or post, it's useful to further determine the stance that indicates whether the author is in favor of or against a specific target entity. For example, Mohammad et al. (2017) created the first stance dataset in Twitter, and proposed a stance detection system using Support Vector Machine (SVM) classifiers with character and word n-grams and word embedding features. But the target entity needs to be specified before determining the stance, and it's only based on the tweet content. It's closely related to aspect-based sentiment analysis tasks in SemEval 2014 (Pontiki et al., 2014) and SemEval 2015 (Pontiki et al., 2015).
In addition to the typical stance detection of texts, social media data are often used in determining the polarization in political opinions (Conover et al., 2011) and predicting voting intentions or outcomes in elections (Tumasjan et al., 2010). Instead of detecting the stance of a single user on a specific target, it's useful to derive the stance from the general public on a topic, which we called topical stance. For example, in SemEval 2016 topical stance detection contest, MITRE (Zarrella and Marsh, 2016) used LSTM with Word2Vec word embeddings. DeepStance (Vijayaraghavan et al., 2016) used CNN models, while Du et al. (2017) used attention models. Dey et al. (2018) developed a two-phase solution to topical stance detection for Twitter including subjectivity detection and sentiment classification using LSTM with attention. Samih and Darwish (2021) proposed user-level stance detection using The 33rd Conference on Computational Linguistics and Speech Processing (ROCLING 2021) . The Association for Computational Linguistics and Chinese Language Processing only a few tweets for users by fine-tuning contextualized embedding. As mentioned in the literature (ALDaye and Magdy, 2021), there are usually two levels of stance detection: statementlevel, which is simply based on text content, and user-level, which is to predict the stance of a user on the target. Also, there could be three different types of stance detection according to targets: target-specific stance, multi-related target stance, and claim-based stance. In addition to stance detection, research on stance prediction is usually concerned with detecting stances before the event.
Most previous studies investigated the micro-level prediction, which estimates the individual user's viewpoint toward a target. For example, Dong et al. (2017) considers joint modeling of content and social interactions for user stance prediction. Darwish et al. (2017) used content and user interactions to calculate user similarity for stance prediction. In this paper, we propose a macro-level approach to stance prediction by aggregating topical stances from post-centric and user-centric views in social media, based on sentiment classification results on very short texts using word embedding features and machine learning methods.

The Proposed Method
There are three major modules in the proposed approach: concept expansion, opinion analysis, and stance aggregation. The overall architecture of the proposed approach is illustrated in Fig. 1: As a preprocessing step, given the topical keyword, topic-relevant posts and comments are collected and represented by word embeddings. First, concept expansion is done by clustering the topic-relevant posts and comments, and extracting the keyphrases in each cluster. Then, machine learning methods are utilized to train the classifiers for sentiment classification. Sentiment orientation is then used to calculate the corresponding stances from both post-centric and user-centric points of view. Finally, the stances are aggregated by their linear combination. In the following subsections, we will explain the details.

Data Representation
Social media contents might be very diverse and noisy. To facilitate more efficient analysis, we routinely crawled all data from the target source media and extracted the corresponding structures from the post-centric and user-centric views and stored in a search engine called Apache Solr for efficient search and analysis. The two different views are described as follows.
In post-centric views, each post consists of the major content and responses from others including replies (or comments), ratings (such as like/dislike), and sharing (such as forwarding, or retweeting, depending on the social platform). These various responses constitute how people think about this post. Generally, the more positive feedbacks a post gets, the more positive it is regarding the topic.
In user-centric views, each user might post an article, and respond to other users' posts, including replies, ratings, and sharing. From these posts and responses, we might be able to observe what he or she thinks about a topic. The more positive comments a user gives, the more positive the user is regarding the topic.
Since we focus on the analysis of text contents and users, we need to correctly identify person names and the concepts of different entities. In this step, we utilize word segmentation and word embedding for the representation of documents.

Word Segmentation:
The feature units of documents usually include segmented words or word n-grams. In the case of Chinese documents, the definition of words depends on the result of word segmentation since there's no space characters between Chinese characters in a sentence. Usually there are two major problems in word segmentation: ambiguity and unknown words. To resolve the issues, lexicon-based and machine learning methods are often used. The size and quality of the lexicon determines the accuracy of the words segmented. In this paper, we utilize a popular open source tool called Ansj 1 for word segmentation. It's a word-based generative model based on a bi-gram model which is a first-order Markov chain. That is, each character is assumed to be dependent on it previous character. The word candidate that generates the maximum union probability will be selected. Since the first-order Markov chain model might not be able to achieve high recalls for unknown words, a Hidden Markov Model (HMM)-based method (Zhang et al., 2003) is used for identifying out-of-vocabulary words. From our observation, this model can generate better segmentation results for person names.
Word Embedding: After feature units are identified by word segmentation technique, we need to find an appropriate representation for documents. Conventional bag-of-words model is not efficient due to the following reasons. Firstly, it's high dimensional and very sparse. Secondly, word orders are completely ignored, which generates ambiguous semantic meanings. In order to better capture semantics in documents, we utilize word embedding models such as Word2Vec. Through the training of contexts from large amounts of documents, we can better predict the contexts of a word or predict a word from its context. Also, it's fixed dimensional which make the machine learning algorithms easier to calculate. Specifically, we represent a document dj by its component words w1, …, wn after word segmentation as follows.
( 1) where V(wi) is the vector representation of each word wi.

Concept Expansion
People might describe the same idea in different terms. Given a single term, the semantics are usually limited. For example, people searching for information about "presidential election" might be interested in the candidates, their names, and election results. To understand what people think about a topic, we need to collect their opinions on all related concepts. In this paper, we utilize document clustering and keyword extraction for concept expansion. Firstly, initial topic was used to 1 https://github.com/NLPchina/ansj_seg collect related documents and grouped into clusters. Then, keywords are extracted from each cluster and the top-frequent keywords are kept as the major concepts. In order to improve the informativeness of the concepts extracted, we repeat the same process by using these keywords to collect related documents for augmenting the keywords until it converges to the number of concepts we need.

Document Clustering:
To obtain all related concepts, we first start with the topic word t. By using search engines such as Google, we get the search result pages P(t). Then, we use the same word embedding models to represent each document pi in its vector form V(pi), from which Kmeans clustering algorithm is used to separate them into K groups. These correspond to the different groups of documents for different concepts. The selection of K depends on how many possible concepts might be related to this topic. In the example of presidential election, the number of clusters K might correspond to the different groups of candidates in the election.
Keyword Extraction: After documents are grouped by their embeddings, the next issue is how to identify the corresponding concepts for each group. Firstly, we apply the same word segmentation technique Ansj on all documents in each cluster to identify the corresponding keywords. Then, we need to discover the named entities since they are often the most important candidates for the major concepts. In this paper, we use Stanford Named Entity Recognizer (Finkel et al., 2005) which employed conditional random fields (CRFs) to recognize the named entities in probabilistic ways.

Sentiment Analysis
To understand the opinion orientation of each post, we first train the sentiment lexicon from our training data. Then, we use ELMs (Huang, 2015) to classify the sentiment into positive, neutral, and negative, and compare with a simple baseline Naïve Bayes classifier.
The structure of ELMs is a neural network with single hidden layer. The major difference of ELMs from common neural networks is its lack of back The 33rd Conference on Computational Linguistics and Speech Processing (ROCLING 2021) Taoyuan, Taiwan, October 15-16, 2021. The Association for Computational Linguistics and Chinese Language Processing propagation phase to reduce training errors. Thus, it's much faster than conventional neural networks. The architecture of ELMs is shown in Figure 2. As shown in Figure 2, ELMs need numeric data as input just like common neural networks, we apply the word embedding models such as Word2Vec on each document. The number of neurons in input layer corresponds to the dimension of word embeddings. The number of neurons in output layer is one, which simply classifies each document as positive or negative. In this paper, the number of neurons in the hidden layer is set as 200.
User ratings might not reflect the actual sentiment orientation of users, for example, in the case of sarcasm. From our observation, people are more proactive in negative ratings, and they might give negative replies or comments with a positive rating. This is possible in some cases where people respond to some people or events instead of the document itself. It was also indicated in related work (Heath, 1996). In order to fix this phenomenon, we adjust the user ratings by combining with sentiment classification of replies or comments as follows.
(2) Where r(dj) is the user rating such as like or dislike, and Sent(dj) is the sentiment orientation of the document.

Stance Aggregation
Given user input topic t and the number of concepts K, we obtain related concepts Q1,…, QK. For each concept Qi, we obtain the set of all the related documents Di and the set of all the related users Ui.
Then, we conduct analyses in two different views as follows.
Post-Centric Stance: For a given concept Qi, we have the set of all related documents Di. For each document dj in Di, instead of using the sentiment orientation defined previously, we first calculate the aggregate score from all the comments. The idea is that: the orientation of a post is determined by the orientation of its comments. (3) Where Comm(dj) denotes all the comments pk for post dj, and class(pk) is defined as in Eq.(2). The higher the score, the more positive people judge on this post.
To accumulate all the scores into the overall post stance for the concept Qi, we define the postcentric stance as follows: (4) where Di is the set of all documents related to concept Qi.
User-Centric Stance: For a given concept Qi, we also have the set of all related users Ui who posted or comments on posts in related concepts. For each user uj in Ui, we consider the aggregate score from all the posts generated by him or her. The idea is that: the stance of a user is determined by the orientation of his/her posts. Where Posts(uj) denotes all the posts pk by user uj, and class(pk) is defined as in Eq.(2). The higher the score, the more positive people judge on this user.
To accumulate all the scores into the overall user stance for the concept Qi, we define the usercentric stance as follows: (6) where Ui is the set of all users related to concept Qi.
Aggregate Stance: For each given concept Qi, the data size might be different in terms of related posts and users. To give a more balanced aggregation, we can further consider the weights for posts and users as follows. (7) Where Di is the set of all documents related to concept Qi. (8) Where Ui is the set of all users related to concept Qi. Thus, the weighted post-centric and usercentric stances for a given concept Qi can be defined as follows: Since we calculate the post-centric and usercentric stances individually, to further allow for the relative importance between the two views, we finally assign a weight for linear combination for the total stance as follows. (11) The idea is that: the higher the total stance for a concept, the more positive people give feedbacks to this concept.

Experiments
In our experiments, we designed our customized crawler in Telnet to collect data from the most popular online discussion forum called PTT. During Feb. 2015 an Jun. 2016, a total of 881,322 documents in Chinese was collected in the discussion board of "Gossip". The number of users participated in these posts is 60,018.

The Effects of Concept Expansion
To verify the effects of concept expansion, we selected a number of topics. The results of concept expansion are as follows:  Table 1: Example results of concept expansion.
As shown in Table 1, we can see more relevant concepts can be extracted. For example, for 2016 presidential election, the candidates and the running mates can also be discovered. In the case of Ma-Xi meeting, the major participants from both sides including the Minister of the Mainland Affairs Council Hsia Li-yan and Taiwan

The Effects of Sentiment Analysis
After concept expansion, we need to conduct sentiment analysis for text documents. We selected a number of concepts to test the performance. The ground truth is taken from user ratings such as likes or dislikes in each comment. The results of sentiment analysis using Naïve Bayes are as follows:  Table 2: Example results of sentiment analysis using Naïve Bayes.

Concept
As shown in Table 2, we can see a good average accuracy of 0.741 and F-score of 0.759 for Naïve Bayes. However, since data is imbalanced, the precision value of some concepts are as low as 0.464. This is not stable. Next, we show the results of sentiment analysis using ELMs.  Table 3: Example results of sentiment analysis using ELMs. Table 3, we can see an average accuracy of 0.655 and F-score of 0.713 for ELMs. Comparing to Naïve Bayes, we can see lower accuracies, but more stable precision values and Fmeasures across different concepts. Given more training data, NB is able to learn the probabilistic distributions. ELM cannot reduce the error with back propagation, which gives much lower recalls. The precision values are only slightly affected. We will analyze the reasons as follows.

As shown in
There are several possible reasons for the mismatch between user ratings and post content sentiments.
The first possible case of incorrect classification is a "false positive". There are many cases when the post content explains the support of one new candidate, but the opinions are against the current officers. That's why we see positive user ratings (for the new candidate), but negative content sentiments (against the current officers). If we conduct sentiment analysis on the contents, they are correctly classified as negative, which is different from the ground truth of positive.
The second example case of misclassification is when a government agency post content criticizing candidate Tsai. Users gave negative ratings against this government post, but positive content in favor of the candidate. That's another type of mismatch for "false negatives".
To show the effects of these misclassification, we selected a part of the posts from the same concepts and manually adjust the labels of two types of misclassified instances.  Table 4: Performance comparison of NB and ELM before and after adjustment of Type-1 errors. Table 4, we can observe the performance improvement for NB in terms of accuracy. Specifically, since false positives are greatly reduced for both NB and ELM, precision values are greatly improved. At the same time, false negatives are increased much more for ELM, which gives lower recall. The best performance can be seen for NB after adjusting Type-1 errors.

Method
Recal l  Table 5: Performance comparison of NB and ELM before and after adjustment of Type-2 errors.
As shown in Table 5, we can observe the performance improvement for both NB and ELM. Specifically, since false negatives are greatly reduced for both NB and ELM, recall values are greatly improved. At the same time, false positives are increased, which gives lower precision. Although ELM can also be improved, the best performance can be seen for NB after adjusting Type-2 errors.
These results are simply sampled from selected topics, it could not reflect the overall performance. But we can see the advantage of adjusting Type-1 and Type-2 errors, which are very common in social media posts, especially for the discussion forum PTT. The sarcastic phenomenon on political issues among social network users have much impact on the sentiment analysis results.

The Effects of Post-centric vs. Usercentric Stance Detection
In this experiment, we want to verify the effects of post-centric and user-centric stance detection. Here, we focus on the prediction of 2016 presidential election in Taiwan by the two views of stance detection using NB and ELM classifiers and manually adjusted ratings, which are denoted as Post-NB, Post-ELM, User-NB, User-ELM, respectively. Then, we have two baselines: Post-Baseline, and User-Baseline, which simply use statistics of user ratings as the baseline for postcentric and user-centric, respectively.
In this experiment, the topic "presidential election" can be expanded into the three candidates, who got the percentages of final votes: Chu-Wang (31.04%), Tsai-Chen (56.12%), and Soong-Hsu (12.84%). These are considered as the ground truth. Firstly, we compared the mean absolute error (MAE) as follows.  Table 6: Performance comparison of election result prediction for both post-centric and usercentric views.

MAE-Soong
As shown in Table 6, we can see the best postcentric result is Post-ELM with a MAE of 2.83%, and the best user-centric result is User-NB with a MAE of 0.79%. For each method, we can obtain better performance for user-centric views.

The Effects of Stance Aggregation
Next, we further determine the stance aggregation using different weights for postcentric and user-centric results. We compared the better results as shown previously with the aggregated results. From our observation, better MAE values can be obtained when is 0.7-0.9, we show the result when = 0.7 as follows.

MAE -avg.
Aggregate-Baseline  Table 7: Performance comparison of election result prediction when = 0.7 in stance aggregation.
As shown in Table 7, we can observe the best performance for ELMs in predicting the percentage of votes for three candidates. Specifically, when aggregating stances using ELMs, the best MAE of 0.52% can be obtained. This shows the potential of the proposed approach to topical stance aggregation from post-centric and user-centric sentiments.

Discussions
From our experimental results, there are some observations: Firstly, from our observations on sentiment classification of PTT data, we found Type-1 and Type-2 errors that frequently occurred in posts. Users might give positive ratings with negative contents, or vice versa. After adjusting these errors, the performance of sentiment classification can be improved for both ELM and NB. Secondly, we consider two different views of stance detection: post-centric and user-centric. User-centric stance detection works better than post-centric, especially for Naïve Bayes.
Finally, we validated the effects of stance aggregation by the weighted sum of both user-centric and post-centric stances. The best prediction performance with MAE of 0.52% can be obtained. It shows the potential of our proposed approach to stance prediction.

Conclusions
In this paper, we have proposed to aggregate postcentric and user-centric sentiments from social media for stance detection. Firstly, we performed concept expansion to obtain the related concepts The 33rd Conference on Computational Linguistics and Speech Processing (ROCLING 2021) . The Association for Computational Linguistics and Chinese Language Processing for the given topic. Secondly, we trained classifiers such as Naïve Bayes and Extreme Learning Machines for sentiment classification. Finally, we proposed a potential way of calculating the individual influences from comments for posts and posts from each user, and aggregating to obtain the total stance for the topic. From our experimental results, we can see a good performance with the best MAE of 0.52% when we aggregate stances estimated using ELMs. This shows the potential of our proposed approach in topic-specific opinion mining and stance detection. Further investigations are needed to evaluate our proposed approach in different topic domains.