Detection of Illicit Drug Trafficking Events on Instagram: A Deep
  Multimodal Multilabel Learning Approach
  Social media such as Instagram and Twitter have become important platforms
for marketing and selling illicit drugs. Detection of online illicit drug
trafficking has become critical to combat the online trade of illicit drugs.
However, the legal status often varies spatially and temporally; even for the
same drug, federal and state legislation can have different regulations about
its legality. Meanwhile, more drug trafficking events are disguised as a novel
form of advertising commenting leading to information heterogeneity.
Accordingly, accurate detection of illicit drug trafficking events (IDTEs) from
social media has become even more challenging. In this work, we conduct the
first systematic study on fine-grained detection of IDTEs on Instagram. We
propose to take a deep multimodal multilabel learning (DMML) approach to detect
IDTEs and demonstrate its effectiveness on a newly constructed dataset called
multimodal IDTE(MM-IDTE). Specifically, our model takes text and image data as
the input and combines multimodal information to predict multiple labels of
illicit drugs. Inspired by the success of BERT, we have developed a
self-supervised multimodal bidirectional transformer by jointly fine-tuning
pretrained text and image encoders. We have constructed a large-scale dataset
MM-IDTE with manually annotated multiple drug labels to support fine-grained
detection of illicit drugs. Extensive experimental results on the MM-IDTE
dataset show that the proposed DMML methodology can accurately detect IDTEs
even in the presence of special characters and style changes attempting to
evade detection.
