Exploring Sequence Feature Alignment for Domain Adaptive Detection
  Transformers
  Detection transformers have recently shown promising object detection results
and attracted increasing attention. However, how to develop effective domain
adaptation techniques to improve its cross-domain performance remains
unexplored and unclear. In this paper, we delve into this topic and empirically
find that direct feature distribution alignment on the CNN backbone only brings
limited improvements, as it does not guarantee domain-invariant sequence
features in the transformer for prediction. To address this issue, we propose a
novel Sequence Feature Alignment (SFA) method that is specially designed for
the adaptation of detection transformers. Technically, SFA consists of a domain
query-based feature alignment (DQFA) module and a token-wise feature alignment
(TDA) module. In DQFA, a novel domain query is used to aggregate and align
global context from the token sequence of both domains. DQFA reduces the domain
discrepancy in global feature representations and object relations when
deploying in the transformer encoder and decoder, respectively. Meanwhile, TDA
aligns token features in the sequence from both domains, which reduces the
domain gaps in local and instance-level feature representations in the
transformer encoder and decoder, respectively. Besides, a novel bipartite
matching consistency loss is proposed to enhance the feature discriminability
for robust object detection. Experiments on three challenging benchmarks show
that SFA outperforms state-of-the-art domain adaptive object detection methods.
Code has been made available at: https://github.com/encounter1997/SFA.
