{'fraction_non_alphanumeric': 0.028619936939121998, 'fraction_numerical': 0.0019403347077370846, 'mean_word_length': 4.53489932885906, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A Hub-and-Spoke Model for Content-Moderation-at-Scale on an Information-Sharing Platform

One of the most expensive parts of maintaining a modern information-sharing platform (e.g., web search, social network) is the task of content-moderation-at-scale. Content moderation is the binary task of determining whether or not a given user-created message meets the editorial team's content guidelines for the site. The challenge is that the number of messages to check scales with the number of users, which is much larger than the number of moderator-employees working for the given platform. We show how content moderation can be achieved significantly more cheaply than before, in the special case where all messages are public, by effectively platformizing the task of content moderation. Our approach is to use a hub-and-spoke model. The hub is the core editorial team delegated by the management of the given platform. The spokes are the individual users. The ratings of the editorial team create the labels for a statistical learning algorithm, while the ratings of the users are used as features. We have implemented a primitive version of this algorithm into our open-source DimensionRank code base, found at"thinkdifferentagain.art".


Introduction
One of the most expensive and labor-intensive parts of running a modern web search or social network service is the implementation of contentmoderation-at-scale. By this, we mean the removal of messages that are deemed in violation of the platform's content guidelines. Indeed, if a network has millions or billions of users, and only tens, hundreds or thousands of employees, exhaustive manual annotation of each message is impossible. It is empirically much more difficult and costly to scale an employee-based organization, than to scale a social media user-base.
The expense of manual content moderation, to the extent this is possible at all, is a burden on large companies. But, even worse, it is a prohibition against the formation of new information-platform companies, because this investment in moderation will not be affordable for small companies.
In this paper, we will show a way to radically lessen the amount of work that needs to be done in order to do content moderation for a network. We call it the hub-and-spoke model of moderation. It crucially leverages statistical methods, especially deep learning, as well as the personal neural representations used in the DimensionRank algorithm and software package (Coppola, 2020).

Information-Sharing Platform
In general, we define a platform as a multi-way channel that connects many producers to many consumers, at the same time. An informationsharing platform is a platform that users use to share information. We refer to a particle of information as a message. Each user on an informationsharing platform is can both author messages or receive them, making each user by turns both a producer and a consumer.

Content Moderation
The platform will have a set of content guidelines, governing whether or not a message is acceptable, meaning it is allowed on the network, or unacceptable, meaning it is not allowed on the network. We assume this is a binary decision, because an information message must either be available on the network, or not. We will call unacceptable messages toxic, and use the variable t. So, for a given message, either t = 1, meaning that the message is toxic, unacceptable and must be removed, or else t = 0, meaning the message is acceptable.

All Messages Public
We assume that each message is broadcast publicly to the entire network. Thus, privacy is assumed to be not an issue. Also, the more times that a message is exposed to users, the more chances users will have to mark it as toxic.

User Participation
We assume that some users participate in the moderation scheme, by being willing to mark messages as either t = 1 or t = 0. We do not require all users to participate in moderation, but this algorithm does require crucially that, for each message, at least one reliable user, and preferably multiple users, will be willing to mark a message as toxic if they see it. In practice, on most existing platforms, we believe that under-reporting has not been a problem, and there has instead been the problem of too many people rating things toxic, rather than too few.

Neural Networks for an Inherently Fuzzy Task
A content moderation policy may in some sense attempt to give "clear rules" for what is a violation or not. However, policies that reference the linguistic or visual human cognitive systems must inevitably refer to topics that require a judgment decision that cannot be written in a rule-based way. This is because rule-based natural language processing and rule-based vison-processing both do not work. Both fields require some amount of machine learning. Thus, we believe it appropriate to model task of content-moderation using a statistical process, e.g., a neural network, which can operate based on its own statistical representation of the data.

The Hub-and-Spoke Model
Our approach to radically reduce costs is to use a hub-and-spoke model.

Hubs and Spokes
The essence of the hub and spoke metaphor is that the hub is a central, organizing node in a network system, and the spokes play a role by co-ordinating directly with the hub. This would be as opposed to a completely decentralized network, in which there is no notion of a "central" node. In our case, we need a central node, because this corresponds to the editorial team, whose need to supervise the network is the initial motivation for this work. So, the hub corresponds to the editorial team, and the spokes to the users. As we said, the traditional problem plaguing content moderation is that the number of messages scales with the number of users, which is much larger than the editorial team. But, we turn this crisis into opportunity, by leveraging the activity and intelligence of users. The users and are invited to flag content as toxic. But, we are not so rash as to ban a message because some individual user calls it toxic. We instead leverage algorithms and statistical learning to predict what the editorial team would think, if they had rated it, given its independent characteristics (text and image), and given the certain set of users have rated either t = 1 or t = 0.
Conclusion We only take a message down if either the editorial team literally marks something as toxic, or if the algorithms believe they would do so, with sufficiently high probability.

Labeled Examples
The goal is to label an individual broadcast message m, as either toxic (t = 1), or acceptable (t = 0). There are two ways we can get a label. First, a user (spoke) can label a message. Second, the editorial team (hub) can label a message. The gold label will come from the editorial team. The labels from the users will be used as features.

Feature Sets
The primary input features are (anonymized) identities of the users who have created labels, either t = 1 or t = 0. Because we are using the Dimen-sionRank algorithm, we will be able to represent each user as a representation vector in a multidimensional space. However, the currently implemented partial version of the algorithm simply uses the number of users as a signal to flag manual review. Aside from using social features, we can also represent a message using a neural representation of the text or image itself.

Training and Prediction
A new training example is created each time that the editorial team makes a new label. A new prediction example is created each time that a user makes a new label t = 1. If a message truly is toxic, we expect it to get multiple votes t = 1, and we will create a new prediction example each time. The more votes for toxic a message receives, the more likely we would expect it to be found toxic.

Implementation
We have implemented a primitive version of this algorithm into our open-source Dimen-sionRank codebase, which can be found at thinkdifferentagain.art. Currently, we use the number of times each article has been flagged to prioritize manual review. However, we plan to implement more sophisticated versions, as the number of users grows. We will provide an update on metrics when they are available.