Dual Cross-Attention Learning for Fine-Grained Visual Categorization and Object Re-Identification
Recently, self-attention mechanisms have shown impressive performance in
various NLP and CV tasks, which can help capture sequential characteristics and
derive global information. In this work, we explore how to extend
self-attention modules to better learn subtle feature embeddings for
recognizing fine-grained objects, e.g., different bird species or person
identities. To this end, we propose a dual cross-attention learning (DCAL)
algorithm to coordinate with self-attention learning. First, we propose
global-local cross-attention (GLCA) to enhance the interactions between global
images and local high-response regions, which can help reinforce the
spatial-wise discriminative clues for recognition. Second, we propose pair-wise
cross-attention (PWCA) to establish the interactions between image pairs. PWCA
can regularize the attention learning of an image by treating another image as
distractor and will be removed during inference. We observe that DCAL can
reduce misleading attentions and diffuse the attention response to discover
more complementary parts for recognition. We conduct extensive evaluations on
fine-grained visual categorization and object re-identification. Experiments
demonstrate that DCAL performs on par with state-of-the-art methods and
consistently improves multiple self-attention baselines, e.g., surpassing
DeiT-Tiny and ViT-Base by 2.8% and 2.4% mAP on MSMT17, respectively.