Conditional Context-Aware Feature Alignment for Domain Adaptive Detection Transformer

Abstract

Detection transformers have recently gained increasing attention, due to its competitive performance and end-to-end pipeline. However, they suffer significant performance drop when the test and training data are drawn from different distributions. Existing domain adaptive detection transformer methods adopt feature distribution alignment to alleviate the domain gaps. While effective, they ignore the class semantics and rich context preserved in attention mechanism during adaptation, which leads to inferior performance. To tackle these problems, we propose Conditional Context-aware Feature Alignment (CCFA) for domain adaptive detection transformer. Specifically, a context-aware feature alignment module is proposed to map the high-dimensional context into low-dimensional space, so that the rich context can be utilized for distribution alignment without optimization difficulty. Moreover, a conditional distribution alignment module is adopted to align features of the same object class from different domains, which better preserves discriminability during adaptation. Experiments on three common benchmarks demonstrate CCFA’s superiority over state-of-the-arts.