Socialbots on Fire: Modeling Adversarial Behaviors of Socialbots via
  Multi-Agent Hierarchical Reinforcement Learning
  Socialbots are software-driven user accounts on social platforms, acting
autonomously (mimicking human behavior), with the aims to influence the
opinions of other users or spread targeted misinformation for particular goals.
As socialbots undermine the ecosystem of social platforms, they are often
considered harmful. As such, there have been several computational efforts to
auto-detect the socialbots. However, to our best knowledge, the adversarial
nature of these socialbots has not yet been studied. This begs a question "can
adversaries, controlling socialbots, exploit AI techniques to their advantage?"
To this question, we successfully demonstrate that indeed it is possible for
adversaries to exploit computational learning mechanism such as reinforcement
learning (RL) to maximize the influence of socialbots while avoiding being
detected. We first formulate the adversarial socialbot learning as a
cooperative game between two functional hierarchical RL agents. While one agent
curates a sequence of activities that can avoid the detection, the other agent
aims to maximize network influence by selectively connecting with right users.
Our proposed policy networks train with a vast amount of synthetic graphs and
generalize better than baselines on unseen real-life graphs both in terms of
maximizing network influence (up to +18%) and sustainable stealthiness (up to
+40% undetectability) under a strong bot detector (with 90% detection
accuracy). During inference, the complexity of our approach scales linearly,
independent of a network's structure and the virality of news. This makes our
approach a practical adversarial attack when deployed in a real-life setting.
