Construction of Large-Scale Misinformation Labeled Datasets from Social
  Media Discourse using Label Refinement
  Malicious accounts spreading misinformation has led to widespread false and
misleading narratives in recent times, especially during the COVID-19 pandemic,
and social media platforms struggle to eliminate these contents rapidly. This
is because adapting to new domains requires human intensive fact-checking that
is slow and difficult to scale. To address this challenge, we propose to
leverage news-source credibility labels as weak labels for social media posts
and propose model-guided refinement of labels to construct large-scale, diverse
misinformation labeled datasets in new domains. The weak labels can be
inaccurate at the article or social media post level where the stance of the
user does not align with the news source or article credibility. We propose a
framework to use a detection model self-trained on the initial weak labels with
uncertainty sampling based on entropy in predictions of the model to identify
potentially inaccurate labels and correct for them using self-supervision or
relabeling. The framework will incorporate social context of the post in terms
of the community of its associated user for surfacing inaccurate labels towards
building a large-scale dataset with minimum human effort. To provide labeled
datasets with distinction of misleading narratives where information might be
missing significant context or has inaccurate ancillary details, the proposed
framework will use the few labeled samples as class prototypes to separate high
confidence samples into false, unproven, mixture, mostly false, mostly true,
true, and debunk information. The approach is demonstrated for providing a
large-scale misinformation dataset on COVID-19 vaccines.
