Adaptive Transformers for Learning Multimodal Representations
  The usage of transformers has grown from learning about language semantics to
forming meaningful visiolinguistic representations. These architectures are
often over-parametrized, requiring large amounts of computation. In this work,
we extend adaptive approaches to learn more about model interpretability and
computational efficiency. Specifically, we study attention spans, sparse, and
structured dropout methods to help understand how their attention mechanism
extends for vision and language tasks. We further show that these approaches
can help us learn more about how the network perceives the complexity of input
sequences, sparsity preferences for different modalities, and other related
phenomena.
