Supervised Multimodal Bitransformers for Classifying Images and Text
  Self-supervised bidirectional transformer models such as BERT have led to
dramatic improvements in a wide variety of textual classification tasks. The
modern digital world is increasingly multimodal, however, and textual
information is often accompanied by other modalities such as images. We
introduce a supervised multimodal bitransformer model that fuses information
from text and image encoders, and obtain state-of-the-art performance on
various multimodal classification benchmark tasks, outperforming strong
baselines, including on hard test sets specifically designed to measure
multimodal performance.
