ViDT: An Efficient and Effective Fully Transformer-based Object Detector
  Transformers are transforming the landscape of computer vision, especially
for recognition tasks. Detection transformers are the first fully end-to-end
learning systems for object detection, while vision transformers are the first
fully transformer-based architecture for image classification. In this paper,
we integrate Vision and Detection Transformers (ViDT) to build an effective and
efficient object detector. ViDT introduces a reconfigured attention module to
extend the recent Swin Transformer to be a standalone object detector, followed
by a computationally efficient transformer decoder that exploits multi-scale
features and auxiliary techniques essential to boost the detection performance
without much increase in computational load. Extensive evaluation results on
the Microsoft COCO benchmark dataset demonstrate that ViDT obtains the best AP
and latency trade-off among existing fully transformer-based object detectors,
and achieves 49.2AP owing to its high scalability for large models. We will
release the code and trained models at https://github.com/naver-ai/vidt
