A multimodal deep learning approach for named entity recognition from
  social media
  Named Entity Recognition (NER) from social media posts is a challenging task.
User generated content that forms the nature of social media, is noisy and
contains grammatical and linguistic errors. This noisy content makes it much
harder for tasks such as named entity recognition. We propose two novel deep
learning approaches utilizing multimodal deep learning and Transformers. Both
of our approaches use image features from short social media posts to provide
better results on the NER task. On the first approach, we extract image
features using InceptionV3 and use fusion to combine textual and image
features. This presents more reliable name entity recognition when the images
related to the entities are provided by the user. On the second approach, we
use image features combined with text and feed it into a BERT like Transformer.
The experimental results, namely, the precision, recall and F1 score metrics
show the superiority of our work compared to other state-of-the-art NER
solutions.
