{'fraction_non_alphanumeric': 0.04048780487804878, 'fraction_numerical': 0.017044476327116212, 'mean_word_length': 4.945240532241556, 'pattern_counts': {'":': 0, '<': 1, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Dual Complementary Dynamic Convolution for Image Recognition

As a powerful engine, vanilla convolution has promoted huge breakthroughs in various computer tasks. However, it often suffers from sample and content agnostic problems, which limits the representation capacities of the convolutional neural networks (CNNs). In this paper, we for the first time model the scene features as a combination of the local spatial-adaptive parts owned by the individual and the global shift-invariant parts shared to all individuals, and then propose a novel two-branch dual complementary dynamic convolution (DCDC) operator to flexibly deal with these two types of features. The DCDC operator overcomes the limitations of vanilla convolution and most existing dynamic convolutions who capture only spatial-adaptive features, and thus markedly boosts the representation capacities of CNNs. Experiments show that the DCDC operator based ResNets (DCDC-ResNets) significantly outperform vanilla ResNets and most state-of-the-art dynamic convolutional networks on image classification, as well as downstream tasks including object detection, instance and panoptic segmentation tasks, while with lower FLOPs and parameters.


I. INTRODUCTION
Convolutional neural networks [1] with vanilla convolution operator have been widely used in various computer vision tasks, such as image classification [2], [3], object detection [4]- [6] and instance segmentation [7]. One of the most important characteristics of the vanilla convolution operator is shift-invariance, which enables the network to possess translation equivalence characteristic through local parameter sharing in the spatial domain, and thus reduce the number of parameters. Nonetheless, as every coin has its two sides, the vanilla convolution operator suffers from three main shortcomings. i) A single convolution kernel can only extract one single pattern in the spatial domain, which weakens the operator's representation capacity. To remedy it, hundreds of kernels are necessary for perceiving tasks in complex scenarios, which in turn increases the number of parameters (#Params) and FLOPs (floating point operations). ii) Vanilla convolution is sample-agnostic, that is, the trained kernel performs convolution operations on different inputs in inference phase with the same parameters, which is often not optimal. iii) Vanilla convolution is not good at dealing with features of varying resolutions because the feature jitters caused by resolution changes may lead to misalignments between frozen kernel weights and features. Therefore, multiscale training is a commonly used technique to improve the CNNs' generalization ability on varying resolutions, but this measure greatly increases the training cost.
To efficiently enhance the representation capacity for diverse inputs, researchers proposed several dynamic convolution operators [8]- [14]. Among them, CondConv [10], Dy-namicConv [11] and Dynet [12] generate multiple sets of parameters for each convolution kernel, and then weights them via an attention mechanism according to the input. This type of operator inherits the shift-invariance property of the vanilla operator while being sample-adaptive. However, they are extremely resource-intensive, especially in deep CNNs. Involution [13] and DDF [14] tremendously alleviate these problems by adaptively dealing with diverse patterns in spatial domain. The former adopts the opposite design philosophy from vanilla convolution, where kernel parameters are adaptive in spatial domain but shared in channel domain. The latter further weights the parameters over the channels via an attention mechanism. Nevertheless, these approaches ignore the advantages of shift-invariance of vanilla convolution, which limits model's representation capacity. This motivates us to rethink the designs of dynamic convolutional networks.
Let's first look at a common features extracting scene shown in Fig. 1(a). It visualizes an example where CNN is dealing with zebra stripes. Although the features at each window locations belong to zebra stripes type, the direction, thickness, shape of each one are different. The similar situations also occur in some dense prediction tasks, such as object detection and segmentation. We can observe that there are often both global shared features for all individuals and unique features owned by each one. Taking the scenario in Fig. 1(a) to illustrate, we named them global shift-invariant (GSI) and local spatial-adaptive (LSA) features, respectively. It motivates us to design a convolutional operator illustrated in 1(b) that can properly extract these two types of features. Therefore, in this paper, we propose a novel two-branch dual complementary dynamic convolutional (DCDC) operator, which aims to boost the representation capacity by simultaneously extracting both of the GSI and LSA features. In DCDC operator, the LSA branch enables the network to make specific responses to different local contents. The GSI branch extracts shared features of each individual or position by aggregating global information of input. Note that both Fig. 1. Schematic diagram of feature extraction. As shown in picture (a), the features in each window can be regarded as a combination of two parts, one is unique to individuals and another one is common to all. It motivates us to propose a two-branch dual complementary dynamic convolution operator illustrated in picture (b), which is powered with two types kernels to extract local spatial-adaptive and global shift-invariant features, respectively, and thus boost the representation capacity. In picture (b), (1-3) represent LSA kernels that are different in each sliding locations, and (4-6) represent GSI kernels that keep same parameters in all sliding locations. of the LSA and GSI branches in the DCDC operator are sample-specific, which is advantageous to adapt to different input samples. Besides, benefitted from powerful modeling capabilities, DCDC operator can enable the model to reduce the number of parameters and thus alleviates overfitting.
In summary, the main contributions are: • We for the first time to model the scene features as a combination of LSA and GSI parts, and then build a DCDC operator to deal with these two types of features, which can significantly enhance the representation capacity compared with vanilla and existing state-of-the-art dynamic convolutions. • We propose corresponding substructures for the LSA and GSI branches to achieve a good trade-off between the parameters, FLOPs, and accuracy (see Figs. 4 (a) and (b). Our proposed DCDC-ResNets can achieve a 1.9∼3.8% Top-1 accuracy gain on ImageNet-1K dataset, but only require 60∼70% of both the #Params and FLOPs compared with ResNets. For downstream tasks such as object detection, instance and panoptic segmentation, marked improvements can also be obtained.

A. Static Convolutions
As the core component of CNNs, convolution operator plays an important role in most computer vision tasks. To improve the representation capacity and reduce the computational cost of vanilla convolution, many potential variants have been proposed. Consideing the hundreds of filters is necessary to deal with diverse patterns, grouped convolutions [2], [15] are used to compress features to multiple independent subspaces to reduce the number of parameters and FLOPs. Motivated by grouped convolutions, the more radical depth-wise separable convolutions are proposed to build light-weight deep CNNs named MobileNets [16], [17], which greatly alleviates the problems of over-parameterization and heavy calculation cost caused by the excessive number of filters. By contrast, the point-wise convolution [18] performs mapping only across the channels with a small quantity of parameters and computational cost. In addition, dilated convolution [19] expands the receptive fields of vanilla convolution by inserting holes between consecutive coefficients of the filters, and thus improve the performance of computer vision tasks benefited from long-range dependence modeling, such as semantic segmentation [20], [21]. Although static convolutions have made breakthroughs in various vision tasks, the problem of sampleagnostic weakens their abilities to adapt to different inputs.

B. Dynamic Convolutions
Due to their properties of spatial contents adaptability and input sample specificity, dynamic convolutions can make up for the above-mentioned shortcomings of static convolutions. Common dynamic convolution can be generally classified into two categories: convolutions with dynamic structures and dynamic parameters [22]. The former ones include dynamic depth [23], [24], width [25], [26], and routing [27], [28] types. However, although the dynamic structures type methods can adaptively change the inference graphs with given inputs, they often require specific training strategies, architecture designs, and careful hyper-parameters tuning [22], etc.
The latter methods mainly include [11], [13], [14], [29]- [35], whose key characteristic is to adaptively predict kernel parameters according to different contents. However, these methods usually suffer from excessive computational cost and memory consumption [11], [12], [31], [36]- [39], restricted application scenes [39]- [42], and lack the ability to model features with shift invariance characteristics [13], [14], which limit the applications of them in deep CNNs. Specifically, CondConv [10], DynamicConv [11], Dynet [12] and WeightNet [29] aggregate multigroup predicted parameters via squeeze-and-excitation soft attention [43] to obtain actual parameters for convolution operations. However, predicting several sets of expert parameters also multiplies the demand for computing resources. On the other side, deformable convolutional operator [37], [38] adaptively learns the grid sampling offsets and scale factors for each pixel, but this manner increases the computational cost and thus obviously damages the inference speed of the networks. Besides, CARAFEs [40], [42] aggregate context information to improve the up-sampling operators that originally depend on pixel interpolations. LIP [41] combines self-attention and average pooling to obtain a pooling operator that is specific at each pixel. However, the application scenes of both CARAFEs and LIP are relatively limited. The more relevant methods to our work such as Involution [13] and DDF [14], they achieve attractive performances on computer vision tasks such as image classification, object detection, and instance segmentation. However, these two methods mainly focus on constructing content-based adaptive kernels, but lack kernels with sufficient ability to explicitly model patterns with shiftedinvariance characteristic.

III. METHODOLOGY
In this section, We first briefly review the calculation rule of the vanilla convolution, and then lead to the details of the DCDC operator.

A. Vanilla Convolution
For a given 4-D input feature X ∈ R B×Cin×H×W , where B, C in , H, W represent the sizes of mini-batch, channels, height, and width, respectively. The calculation processes of vanilla 2-D convolution with kernel weight W ∈ R Cout×Cin×k×k and bias B ∈ R Cout can be formulated as follows: where k and C out represent kernel size and the number of output channels, respectively. Y (b,n,h,w) denotes scalar output at the X b C (h,w) cube for the n-th filter, X b C (h,w) ∈ R Cin×k×k represents the sliding feature cube at the (h, w) location of the b-th input sample, here h ∈ [1, H],w ∈ [1, W ], and b ∈ [1, B]. Ω k×k represents the index set of spatial position inside kernel, it can be expressed by where denotes floor function. Ω in is the index set of input feature channels defined as Eq. 1 indicates that the vanilla convolution shares the kernel parameters in all sliding positions, which reveals its shiftinvariant property. Besides, the vanilla convolution uses the same kernel parameters to calculate the output for different input samples, this also indicates its sample-agnostic property. Although shift invariance characteristic enables the operator to capture same or similar features at different locations in spatial domain, it lacks content adaptability. Moreover, the sample-agnostic property also limits the representation ability for different input samples.

B. DCDC
As conceptually demonstrated in Fig 2(a), we design both of the LSA and GSI branches to extract local spatial-adaptive and global shift-invariant features, respectively, and then output the summation of them. It can be formulated as follow: where F lsa and F gsi represent convolution operations of LSA and GSI branches. X and Y denote input and output of the DCDC operator. φ and ϕ represent the sub-networks used to predict the kernel parameters for LSA and GSI branches, and θ and γ are the weights of φ and ϕ, respectively. In addition, the proposed DCDC operator should be lightweight enough to be able to construct deep CNNs. 1) LSA Dynamic Convolution Branch: Inspired by Involution [13], we design the LSA dynamic convolution branch. Instead of optimizing weights via gradient descent, the Involution operator uses an additional sub-network to directly predict kernel parameters according to local contents. The calculation processes of Involution operator with kernel size of k lsa ×k lsa can be formulated as where φ represents the sub-network to adaptively predict the k lsa × k lsa kernel parameters relying on local contents, H (b,h,w) denotes the generated parameters of b-th input sample at (h, w) location. Superscript lsa means local spatialadaptive branch. The definition of the other items remain the same as Sec. III-A. It is observed from Eqs. 5 and 6 that the Involution operator possesses spatial-adaptive and samplespecific characteristics.
The vanilla Involution operator shown in Fig. 3(a) sets φ as two series point-wise convolutional layers and generate H (b,h,w) relying only on a single central pixel (denoted by superscript 1 × 1 in Eq. 6) at the (h, w) position. Predicting k lsa × k lsa kernel parameters based on only a single central pixel is obviously not optimal, it may suffer from insufficient effective receptive field. Note that although the center pixel on the features of deep layer already has a general receptive field much larger than a single pixel with respect to input sample, it is still limitedly benefits the prediction of LSA kernel parameters beyond the center pixel at current layer. In addition, ignoring the connections between neighborhood pixels and performing multi-level mapping only on a single pixel is inefficient for modeling features. To remedy it, as illustrated in Fig. 3(b) (detailed structure shown in upper part of Fig. 2(b)), we skillfully embed depth-wise convolutional layers before each point-wise convolutional layer in φ. It greatly alleviates these problems via expanding receptive fields on the current layer and aggregating information from neighbouring pixels. In summary, the process of predicting LSA kernel parameters can be expressed by where σ dw and σ pw represent depth-wise and pixel-wise convolutional layers, respectively. Here, s×s means the spatial region of pixels used to predict LSA kernel parameters, it should be at least greater than or equal to k lsa × k lsa in LSA branch. The design of alternating depth-wise and pixel-wise layers can substantially expand the receptive fields of φ via modeling features in the spatial and channel domains without significantly increasing the computational cost. Further, this design enables the φ to generate more optimal kernel parameters, and thus improve the representation capacities of LSA branch, which is confirmed in Sec. IV-A4 (ablation experiments).
2) GSI Dynamic Convolution Branch: Although vanilla convolution is shift invariant, the kernel parameters are static and not learned through focusing on global shared features, thus it may not be optimal when sliding over the whole features in spatial domain. In addition, its sample-agnostic property also impairs the representation capacity for different input samples. To remedy this, we propose GSI dynamic convolution branch. On the one hand, it possesses shift-invariant characteristics by specifically aggregating global features to predict more appropriate parameters of GSI kernel than vanilla convolution. In other words, it has better modeling capabilities for the features shared by individuals. On the other hand, it is sample-specific that can flexibly adapt to different input samples. Specifically, we directly use a sub-network to predict the kernel parameters, the entire procedure of GSI dynamic convolution operator with kernel size of k gsi × k gsi can be defined as: where P b ∈ R Cout×Cin×kgsi×kgsi represents the GSI kernel parameters predicted by sub-network ϕ for the b-th input sample from the mini-batch. Superscript gsi means global shift-invariant branch. The definition of the other items remain the same as Sec. III-A. On the one hand, it can be observed from Eq. 8 that the GSI operator applies the same kernel to all sliding windows in spatial domain, but the different ones in channel domain. In contrast, Eq. 5 indicates that LSA operator applies the different kernels in spatial domain, while the same one in channel domain. Therefore, both of these information encoding approaches are dual complementary to each other, and finally enhance the performance of DCDC operator. On the other hand, it is clear from Eq. 9 that the GSI operator owns sample specificity characteristic, and thus can achieve better representation capacity for different inputs than vanilla convolution. The effectiveness of GSI branch is confirmed in Sec. IV-A4 (ablation experiments). Implementation details. As shown in the bottom branch of Figs. 2(a) and 2(b), the design philosophy of ϕ is that it can predict more suitable GSI kernel parameters to extract global shared features only after looking features' panorama. Therefore, for a given feature X ∈ R B×C×H×W , we first map the channel from C to √ C through a point-wise convolutional layer to reduce the computational cost and enhance nonlinearity. Then we aggregate the entire context information via a global average pooling layer, and use another point-wise convolutional layer to obtain the GSI kernel parameters with size of Overall, the parameter predicting process can be formulated as follows: where σ pw and GAP represent the pixel-wise convolution and global average pooling layers, respectively. Finally, after getting the output Y gsi by Eq. 8, we use another pixel-wise convolutional layer to map channels from Fig. 3. Comparisons of vanilla Involution [13] (a) and the posposed LSA operator (b). The green areas in the cubes represent the convolved pixels inside the features. N denotes the number of embedded depth-wise and piexel-wise convolutional layer pairs. ⊗ denotes channel-wise multiplication. Reshape and Expand indicate the operations for reshaping pixel-wise weights to channel-wise ones, and then expanding weights in channel domain. Compared with Involution, the proposed LSA operator can adaptively consider pixel information in a wider range when generating k lsa × k lsa kernel parameters. Therefore LSA operator can achieve better performance than vanilla Involution.
3) DCDC operator: Here, we achieve the final DCDC operator with the formulation: The two terms on the right side of the equation can be obtained from Eq. 5 and 8, respectively. Note that the entire DCDC operator is differentiable, which facilitates us to embed it in the network and optimize end-to-end.

IV. EXPERIMENTS
Based on the proposed DCDC operator, we construct the DCDC-ResNet series backbones and train them from scratch on the ImageNet-1K dataset [44]. Then we evaluate the transfer ability on downstream computer vision tasks including object detection, instance segmentation and panoptic segmentation on MS-COCO dataset [45].
A. Image Classification 1) Datasets and settings: The ImageNet-1K dataset contains 1.28M training and 50K test images with 1K categories. Our training protocal is consistent with [13] based on MMClassification [46] library. To construct the DCDC-ResNet series backbones, we replace 3×3 convolutions in the bottleneck and stem of ResNet by DCDC operator. Considering trade-off between computational cost and accuracy, we set k dw lsa , k gsi and λ hyperparameters (see Fig. 2(b)) of DCDC operator to 3, 1 and 1, respectively, which will be further studied in ablation experiments. α and k lsa are set to 0.25 and 7 adopted from [13]. The mini-batch and initial learning rate with cosine annealing policy are set to 256 and 0.1. All models are trained for 130 epochs via stochastic gradient descent optimizer with the momentum of 0.9 and the weight decay of 0.0001. The learning rate warm-up strategy with linear growth style at first 5 epochs is also adopted to ensure the stability of convergence. In addition, we also train the model with the same protocal with [14] based on the timm [47] library (the training epochs is also 130) to make a fair comparison, the corresponding results are marked with * in Table I. Table I reports the Top-1 accuracy, #Params and FLOPs of DCDC-ResNet-26, -38, -50, and -101. It can be observed that with the same number of layers, our DCDC-ResNet backbone can consistently achieve higher accuracies while with lower FLOPs and #Params compared with the seminal ResNets [3]. Especially, DCDC-ResNet-26 achieves a 3.8% Top-1 accuracy improvement while reducing FLOPs by 29% and #Params by 32%, compared with ResNet-26. DCDC-ResNet-38 even achieves comparable accuracy to ResNet-101, but only requires approximately 28% of the FLOPs and #Params. In addition, DCDC-ResNet series has surpassed many powerful attention-based, group convolution based, etc., variants of ResNet. For example, DCDC-ResNet-50 can achieve 2.3%, 1.9% and 0.9% gains in Top-1 accuracy compared with SENet-50 [43], ResNeXt-50 [15] and Res2Net-50 [54], with only about 50%∼60% #Params and FLOPs. Note that some shallower DCDC-ResNets can even outperform most deeper networks. Such as DCDC-ResNet-38 vs. ResNeXt-50 and Res2Net-50, DCDC-ResNet-50 vs. ResNeXt-101 and Res2Net-101, yet DCDC-ResNets requires only about one-third computational cost. For presentation purposes, we also visualize the relationships between accurary, parameters and FLOPs in Figs. 4 (a) and (b).

3) Comparisons to state-of-the-art dynamic networks:
We also conduct full comparisons with stronger dynamic convolutional backbones. As shown in Table II, the DCDC-ResNets are still obviously superior to most state-of-the-art dynamic convolutional networks. Specifically, DCDC-ResNet-50 outperforms DwCondConv-50 [10], DwWeightNet-50 [29], RedNet-50 [13] and DDF-ResNet-50 [14] by 0.6% ∼ 2.0% Top-1 accuracies with close computational cost. Note that compared with MDDC-50 [30], ODConv-50 [55], etc., DCDC-ResNet-50 can approximately achieve 1.0% Top-1 accuracy, while reducing about 50% #Params and 30% FLOPs, respectively.   Table III, we conduct comprehensive experiments to investigate the effectiveness of each component in DCDC operator. The baseline method is RedNet-26 [13]. The LSA and GSI represents the proposed local spatial-adaptive and global shift-invariant branches. It can be seen that DCDC-ResNet only with LSA branch can achieve an obvious 0.8% accuracy improvement with negligible increases of #Params and FLOPs compared to vanilla RedNet [13]. This proves that predicting kernel parameters based on large-scale information in LSA branch is significantly better than that based on single-pixel regions as discussed in Sec. III-B1 and Fig. 3(b).

4) Ablation experiments: In
we design two contrast experiments to meticulously verify the effectiveness of the GSI branch. One experiment is carried out by replacing the dynamic convolution (i.e., GAP → PW-conv2 → * conv in Fig. 2(b)) in GSI branch with a 3×3 vanilla convolution layer, and the other one is replacing the entire GSI branch with a 3×3 vanilla convolution layer (this one has more feature channels than prior), which are denoted as Conv small and Conv large , respectively. The former can only get a slight 0.2% accuracy increase. Although the latter can obtain a 0.7% accuracy gain, the corresponding #Params and FLOPs are risen significantly. In contrast, the experimental result with GSI branch can achieve a 0.7% accuracy improvement while with only insignificant increases in #Params and FLOPs. These experiments solidly demonstrate the effectiveness of the GSI branch.
In Table IV, we study the effects of (a) the kernel size (k dw lsa ) of the embedded depth-wise convolutions in LSA branch, (b) the kernel size (k gsi ) of the designed dynamic convolution in GSI branch, and (c) the scaling factor λ controlling the number of feature channels in GSI branch on the experimental performance. Here, the baseline method is DCDC-ResNet-26, the corresponding parameters k dw lsa , k gsi , and λ are set to 3×3, 1 × 1, and 1.0, respectively.
(a) Increase k dw lsa while keeping k gsi and λ unchanged. Results show that the performance of network can be significantly improved, while the growth of #Params and FLOPs is almost negligible.
(b) Increase k gsi while keeping k dw lsa and λ unchanged. Results show that the performance gains are marginal.
(c) Increase λ while keeping k dw lsa and k gsi unchanged. Results show that the performance can be greatly improved with marginal computational cost increase.

DCDC-ResNets.
5) Visualizations of LSA and GSI Kernels: In this subsection, to study the modeling ability of the DCDC operator for local spatial-adaptive and global shift-invariant features, we visualize the LSA and GSI kernels, respectively. In Fig. 5, we randomly choose some images from the ImageNet dataset evaluation split to visualize the LSA and GSI kernels of 2ndstage or 3rd-stage of DCDC-ResNet-50. It can be observed that the kernels of the LSA branch own a strong contents-adaptive characteristic, and can sketchily perceive the contour of the objects. Moreover, we can also observe that when encountering features with similar patterns, the LSA branch can generate similar kernels. Because of the consistent response to similar patterns, the network can focus on a larger range rather than some specific local features when predicting the results. This point is also confirmed again in the next subsection, i.e., Sec. IV-A6. In contrast, the kernel of GSI branch is global shift-invariant, which can aggregate the global shared features. Note that different from vanilla convolution, the GSI dynamic operator is sample-specific and has more strong ability to deal with different input samples.
6) Class Activation Maps: To further understand the modeling ability of DCDC operator, we use Grad-Cam [56] to visualize the class activation map (CAM), which is a commonly used technique to reveal the regions of interest for CNNs to make predictions. As shown in Fig. 6, we randomly choose some images to generate the CAMs, which have a redder color in areas with stronger attention. It can be observed that compared to ResNet, the proposed DCDC-ResNet has more consistent and stronger modeling ability for local adaptive features that contain discriminative information of the object, such as texture features on body of 'electric ray'. Benefited from GSI branch and LSA branch with larger receptive field, DCDC-ResNet tends to focus on the whole of the object rather than some specific local features compared with RedNet, such as objects of 'lakeland terrier', 'electric ray' and 'airliner', etc., which enables the network to make more accurate predictions. In addition, DCDC-ResNet can more precisely capture features to recognize hidden objects in complex environments compared with ResNet and RedNet, such as 'monkey' in trees.

B. Object Detection and Instance Segmentation
To verify the generalization of DCDC-ResNets on downstream compute vision tasks, we perform extensive object detection and instance segmentation experiments on MS COCO [45]. We choose Faster RCNN [4], RetinaNet [57], and Mask RCNN [7] as the main benchmarks and conduct comparative experiments based on ResNet [3] backbones and the state-of-the-art dynamic convolutional network of Red-Net [13]. In addition, we also perform experiments on superior benchmarks, such as HTC [58], to further verify the potential of DCDC-ResNets.
1) Dataset and Evaluation Metrics: The MS COCO dataset contains 118K training, 5K validation and 20K test images with 80 categories instances. For convenience, all experiments are evaluated on the validation set. We report COCO-style mean Average Precision of bounding boxes and masks (i.e., AP bbox and AP mask ) with intersection over union (IoU) thresholds from 0.5 to 0.95 with an interval of 0.05. AP S , AP M , and AP L denote AP for small (pixel area s<32 2 ), medium (32 2 ≤ s ≤96 2 ), and large (s>96 2 ) objects, respectively. 2) Implementation Details: Unless otherwise specified, we uniformly train all models for 12 epochs with stochastic gradient descent optimizer based on MMdetection library [60]. The initial learning rate, momentum and weight decay factors are set to 0.02 (0.01 for RetinaNet), 0.9, and 0.0001, respectively. The learning rate is scaled by a factor of 0.1 at the 8-th and 11-th epochs. The 500 iterations linear warmup strategy of learning rate with a factor of 0.001 in first epoch is also adopted to guarantee smoothness of training processing. The shorter and longer sides of input images are set to 800 and 1333. The mini-batchsize is set to 16 with eight GPUs. We carry out comparative experiments based on ResNet-50 [3], RedNet-50 [13] and proposed DCDC-ResNet-50 backbones respectively. All backbones are pretrained on the ImageNet-1K [44] dataset. For HTC, the total epoch is set to 20, and the learning rate is scaled by a factor of 0.1 at the 16-th and 19-th epochs.
3) Main Results: The performance comparisons are shown in Table V. Overall, the proposed DCDC-ResNet enables all benchmarks to achieve significant performance improvements compared with the ResNet baseline and state-of-theart dynamic convolution backbone of RedNet. Specifically, compared with ResNet-50 based detectors, the DCDC-ResNet-50 based ones improve the performance by 2.7 AP and 3 AP, while reducing the #Params and FLOPs by 23%/14% and 25%/12% on Faster RCNN [4] and RetinaNet [57] benchmarks, respectively. Furthermore, if we replace the convolutional layer in neck of detector with DCDC operator, we will get a more noticeable performance promotes of 3.2 AP, and the #Params and FLOPs are further reduced by 28% and 35% on Faster RCNN [4] benchmark. Compared with the RedNet-50 [13], our DCDC-ResNet-50 is still able to improve the performance by about 0.7∼1.3 AP with almost no increase in #Params and FLOPs on RetinaNet [57] and Faster RCNN [4] benchmarks.
DCDC-ResNet-50 outperforms dynamic convolutional network of RedNet-50 by 0.9 AP bbox and 0.7 AP seg . More gratifyingly, DCDC-ResNet-50 obtains 3.9 AP seg and 6.1 AP seg improvements for large scale object segmentation compared with RedNet-50 and ResNet-50, respectively. This fully demonstrates the superiorities of the proposed designs of predicting LSA kernel via considering pixels in a wider range, and predicting GSI kernel via aggregating global features discussed in Sec. III-B1 and Sec. III-B2, respectively. Expermients based on HTC [58] show that DCDC-ResNets are still able to bring the considerable performance gains on superior benchmarks. 4) Visualization of Object Detection and Instance Segmentation: In this subsection, we visualize some object detection and instance segmentation examples based on Mask-RCNN [7] benchmarks. As shown in Fig. 7, we randomly display some visualization example from evaluation split of the COCO dataset. It can be observed that the results based on proposed DCDC-ResNet-50 have lower false positive and higher recall rates compared with that based on ResNet-50 and RedNet-50, which demonstrates the effectiveness of the proposed methods.

C. Panoptic Segmentation
To further evaluate the adaptabilities of DCDC-ResNets, we extra conduct several experiments on the panoptic segmentation task. The more challenging panoptic segmentation task proposed in [61] can be regarded as a combination of object instance and stuff semantic segmentation subtasks. We adopt panoptic feature pyramid networks (PFPN) [59] as the benchmark. Unless otherwise specified, the hyperparameters for all experiments are derived from MMdetection [60] and we train all models for 12 epochs on MS COCO panoptic segmentation dataset [45].
1) Dataset and Evaluation Metrics: In addition to the data of 80 categories things used in object detection and instance segmentation experiments, 53 categories labeled stuffs data are also included to train the models of panoptic segmentation task. All evaluations are performed on the validation set. We adopt standard evaluation metrics proposed in [61] including PQ (panoptic quality), SQ (segmentation quality), and RQ (recognition quality), and report the performance on things and stuffs marked with Th and St superscripts, respectively.
2) Main Results: Table VII compares the performances of the proposed DCDC-ResNet with those of ResNet [3] and RedNet [13]. It can be observed that the DCDC-ResNet-50 can remarkably achieve 2.8 PQ, 0.8 SQ and 3.2 RQ improvements respectively compared with ResNet-50 baseline, while costing 21% fewer #Params. If we replace all convolution layers in the neck of framework with DCDC operator, 3.0 PQ, 1.1 SQ and 3.3 RQ boosts can be further achieved. In addition, compared with the state-of-the-art dynamic convolutional backbone of RedNet-50, our methods still obtain 0.7 PQ, 0.2 SQ and 0.8 RQ improvements with almost no increase of parameters. Note that the experimental results with DCDC-ResNet-50 exceed that with ResNet-101 (growths of 1.1 PQ, 0.6 SQ and 1.3 RQ), while the #Params is only about half of it (34.3 M vs. 64.9 M).
V. CONCLUSION In this paper, we novelly model the features as a combination of local spatial-adaptive and global shift-invariant parts, and then propose a two-branch dual complementary dynamic convolution operator to properly deal with these two types of features, which significantly enhance the representation capacity. The DCDC-ResNets built based on the proposed operator achieves obviously better performance than ResNet baseline and most of the state-of-the-art dynamic convolutional networks, while with fewer number of parameters and FLOPs. We also perform transfer experiments on downstream vision tasks including object detection, instance and panoptic segmentations to evaluate the generalization ability of model, the experimental results have shown remarkable performance improvements.