{'fraction_non_alphanumeric': 0.03485560154397061, 'fraction_numerical': 0.011417011579779566, 'mean_word_length': 5.053913288288288, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Counting with Adaptive Auxiliary Learning

This paper proposes an adaptive auxiliary task learning based approach for object counting problems. Unlike existing auxiliary task learning based methods, we develop an attention-enhanced adaptively shared backbone network to enable both task-shared and task-tailored features learning in an end-to-end manner. The network seamlessly combines standard Convolution Neural Network (CNN) and Graph Convolution Network (GCN) for feature extraction and feature reasoning among different domains of tasks. Our approach gains enriched contextual information by iteratively and hierarchically fusing the features across different task branches of the adaptive CNN backbone. The whole framework pays special attention to the objects' spatial locations and varied density levels, informed by object (or crowd) segmentation and density level segmentation auxiliary tasks. In particular, thanks to the proposed dilated contrastive density loss function, our network benefits from individual and regional context supervision in terms of pixel-independent and pixel-dependent feature learning mechanisms, along with strengthened robustness. Experiments on seven challenging multi-domain datasets demonstrate that our method achieves superior performance to the state-of-the-art auxiliary task learning based counting methods. Our code is made publicly available at: https://github.com/smallmax00/Counting_With_Adaptive_Auxiliary

. Overview of the proposed network structure in the scene of crowd counting. An attention-enhanced adaptively shared backbone network is proposed to enable both task-shared and task-tailored features learning. A novel Graph Convolution Network (GCN) reasoning module is introduced to tackle issues of cross-domain features reasoning among three different tasks. A novel loss function L DCD is proposed to take into account more adjacent pixels for regional density difference, which strengthens the network's generalizability.
showed that a density map regression paradigm gains satisfying results. In these methods, given an input image, a CNNbased network is used to regress the corresponding density map; the summed pixel value of the density map gives the total counting numbers in that image. However, there is still much room to improve the counting performance due to challenging issues [1], such as significant scale changes, wide variations of density levels, and complex scene backgrounds. To solve these problems, some previous methods [7], [8], [9], [10], [11] relied on different types of information granularity in terms of 'auxiliary task learning'. These methods applied a single shared backbone network structure to extract generalized features for all the tasks. Unfortunately, this strategy may lead to under-fitting as the generalizable representation often cannot effectively describe the comprehensive cross-domain features across different tasks at the same time [12], [13]. Intuitively, our motivation is that the backbone network should be able to yield both generic (or universal) representations shareable for all the tasks and specialized features tailored for individual tasks. To this end, we designed an attention-based adaptive shareable backbone network to enable task-shared and taskcustomized features learning in an end-to-end manner ( Fig.  1 shows an overview of the network architecture). Note that, the term 'auxiliary task learning' is referred to as the feature learning of different density information granularity levels. Specifically, the crowd segmentation task and the density level segmentation task in Fig. 1 are the auxiliary tasks, and the density map regression task is the main task. We generated the ground truth of crowd segmentation and density level segmentation tasks from the density map regression task's ground truth. Intuitively, the information from the ground truth of auxiliary tasks is not increased; however, the information is enhanced and specified through auxiliary tasks in terms of different density information granularity.
Instead of aiming to achieve the best performance for all the tasks, our adaptive shareable backbone network primarily focused on optimizing the primary density map regression task, along with the multi-granularity information enhancement from the other auxiliary tasks. Our backbone network contained a multi-level information aggregation mechanism to iteratively and hierarchically fuse features learned from different stages and auxiliary branches to tackle large scale changes problems. We also applied two attention-based auxiliary task learning branches: 1.) crowd segmentation task to indicate spatial regions of interest to tackle complex background issues; and 2.) density level segmentation task to be aware of the varied density levels across the image, which can tackle the significant variations of density problems. An ablation study demonstrates that our adaptive auxiliary branches improve performance over a single-shared backbone based auxiliary task learning network with comparable model size.
Apart from the aforementioned components, we also studied how to reason and fuse features from different tasks for density map regression. The features extracted from crowd segmentation and density level segmentation branches belong to different feature domains with various granularity representations. Direct fusion (e.g. element-wise multiplication or channel-wise concatenation) among three task branches' outputs can result in domain conflicts [14]. Thus, further reasoning is necessary to improve the counting performance. To this end, we exploited the information reasoning nature of Graph Convolutional Networks (GCN). GCN has recently shown promising reasoning ability on many computer vision tasks, such as scene understanding [15], image segmentation [16], [17], etc., but has been rarely studied in the crowd counting task domain. Our model projects a collection of spatial-aware density feature map's pixels with similar density levels to each graph vertex and exploits a GCN to reason about the relations among graph vertices. This is different from a recent work [14], which directly treated cross-domain feature maps as graph vertices and utilized a cascaded Graph Neural Network (GNN) to reason the cross-scale relationships. Our experiment results have proved that the proposed GCN reasoning module helps to improve the counting accuracy.
We also proposed a novel loss function to supervise the main task learning processes. Notably, for the supervision of density map regression, the widely adopted Least Absolute Error (L1) or Least Square Error (L2) loss in previous counting methods [18], [7], [19], [20] assumes pixel-wise independence, which supervises the predicted density map based on the individual pixels. However, it has two significant weaknesses. Firstly, the predicted density map tends to be over-smooth [8]; specifically, it may underestimate high-density level regions and overestimate low-density level regions. As a result, the model may primarily focus on achieving lower count errors rather than regressing high-quality density maps; thus, it cannot reflect the actual density levels. Secondly, without a large receptive field, individual pixel-wise loss functions may ignore the regional density level information during the training process [21]. Unbalanced low and high-level density distributions may introduce significant bias in the training process, thus weakening the network's robustness. To address these issues, we proposed a novel loss function for density map regression, called Dilated Contrastive Density Loss (L DCD ), where the density difference among dilated adjacent pixels is utilized to provide additional regional supervision. Ablation studies demonstrate that our proposed regional loss function can improve the counting performance of the pixel-wise loss supervised methods.
In summary, this work makes the following contributions: 1.) We addressed the feature learning issues of the backbone network of the auxiliary task in crowd counting challenges, by enabling task-shareable and task-specified feature learning simultaneously with a primary focus on the main task. 2.) We proposed crowd segmentation and density level segmentation as auxiliary tasks in crowd counting with additional spatial crowd location and density level information enhancement. Moreover, a GCN model was proposed to reason about the cross-domain feature relations between density map regression and other auxiliary tasks. 3.) We proposed a novel loss function tailored for density map regression, strengthening the network's generalizability and improving the counting accuracy. 4.) We conducted extensive experiments on seven well-known challenging counting benchmarks. Quantitative and qualitative results demonstrated that our model achieves state-of-the-art performance. Especially, to the best of our knowledge, we achieved the best counting performance among auxiliary tasks based counting methods on NWPU-Crowd [22] benchmark 1 , which is currently the largest crowd counting benchmark. Our model is robust and generalizable to indicate the wrong labeled or miss labeled object in the test datasets. Please refer to Section V-D for more details.

II. RELATED WORK
In recent years density map regression-based counting methods with CNN achieved good performance. For example, Boominathan et al. [23] proposed a dual-column network to combine low-level and high-level features in different layers to estimate the count. However, because of the conflicts from optimization among different columns [24], these types of network structures have difficulty in attaining global minimization. Other works employed single column network structures and handled different scale challenges [25], [26], [27], [28], [29], [30] with adaptive modules, such as scaled spatial pyramid pooling [19], [31], [32] or Dilated kernels of filters [33], [34], [35], [36], [37]. They achieved promising counting performance along with architectural simplicity and training efficiency.

A. Attention-Based Counting
The visual attention mechanism was applied among several works [38], [21], [39], [40], [41], [42], [36] in the crowd counting task, which helped the network focus on valuable information and addressed several challenges. For example, Miao et al. [38] utilized a shallow feature based attention module to highlight the regions of crowd interest and filter out the noise in the background clutter. To tackle various density levels issues, Jiang et al. [21] employed an attention mask to refine the density map for adapting to different density levels. Furthermore, Zhang et al. [39] proposed the Attention Neural Field that incorporated non-local attention modules and conditional random fields to maintain multiscale features and long-range dependencies, to handle large scale changes problems of the input crowd images. Wan et al. [43], [36] exploited the self-attention mechanism to adaptively generate density maps with different Gaussian kernel sizes, which is used as the ground truth to supervise the model. The aforementioned methods adopted the attention mechanism as a feature enhancement module to implicitly address the crowd counting task's challenges, such as significant scale changes, wide variations of density levels, and complex scene backgrounds. Our model explicitly addressed those challenges through auxiliary tasks. On the other hand, our model adopted the attention mechanism to construct an adaptively shared backbone network, enabling the task-shared and task-specific features learning simultaneously.

B. Auxiliary Tasks Based Counting
Recently, auxiliary task learning based counting methods [44], [45], [46], [47], [48], [49], [50], [51], [9], [52], [53] attracted researchers' attention because of its ability to capture extra granularity information and contextual dependencies for the density map regression. Most of the methods utilized the potential of a model itself with auxiliary tasks, such as object detection, crowd segmentation, density level classification, etc., to enhance the feature tuning for density map regression. For example, the task of patch-based density level classification [7], [54], [55], [9], [56], [57], [58] can enhance patchlevel density level information, which helped to address the underestimation and the overestimation problems of density map regressions. However, it may be difficult to guide the pixel-wise density map regression via patch-wise density level classification because of the gap between pixel-level and patch-level feature learning. In contrast, our model proposed a density level segmentation auxiliary task, which can be regarded as the dense pixel-wise density level classification task. In this way, our model can enhance the pixel-wise density level information to the pixel-wise density map regression task, aiming to address the challenges of wide variations of density levels.
Moreover, because the background regions in complex scenes contain confusing objects or similar appearance, the crowd segmentation task, adopted by previous methods [59], [7], [14], [10], [60], can provide spatial location information for the crowd, which highlighted the foreground over the background and guided the network focus on the region of interest. Our model also adopted the crowd segmentation task because of its superiority in spatial location information enhancement. Similarly, the task of object (crowd) detection [8], [61], [62], [63], [64], [65] can enhance location information and alleviate local spatial inconsistency issues in the density map.

C. Learn to Count with Different Supervisions
Instead of tackling the counting task through different learning frameworks or strategies, recent methods [66], [67], [68], [69], [70], [71], [72], [73], [74], [75] payed attention at the way of supervisions. For example, Sravya et al. proposed a bin loss [68] to enable the data-distribution aware optimization, which helped to address the domain variation challenges from different crowd data source. Song et al. [69] studied the counting problem in a different way, where a combination of Euclidean loss and Cross Entropy loss was used for point locations learning, instead of density map regression. Along the same line, Bayesian loss was proposed by [71] to provide more reliable supervisions at each annotated point. Differently, Wan et al. [70] studied the combination of pixel-wise loss and point-wise loss, which investigated the density map representation through an unbalanced optimal transport problem. [72] proposed a novel loss function to address the spatial annotation noise during training, where a weighted MSE term and a pixel-wise correlation term were involved. Recently, [73] proposed distribution matching loss to tackle the weakened generalizability of the Gaussian smoothed density map. Moreover, Wang et al. [74] treated the counting with density map as a classification problem, where a Cross-Entropy loss was used to classify each patch into certain intervals.
The aforementioned methods introduced different loss functions to supervise the model, such as points location, bounding box, matching, ranking, classification, etc.. However, the mainstream counting methods still relies on pixel-wise supervision with the density map ground truth [1], such as L1 or L2 loss functions. In this work, we proposed a Dilated Contrastive Density Loss (L DCD ) to improve the pixel-wise loss's receptive field and increase the regional supervision.

A. Ground Truth Generation
Following [76], given a set of N images is generated by filtering the points with a normalized Gaussian kernel. The total object count number T i of image I i can be attained by summing all pixel values of the density map D i .
The ground truth mask of the crowd segmentation task is generated from the density map ground truth. Given a set of is set to 1 if the pixel valve in the density map is larger than the zero and 0 otherwise.
The ground truth mask used by the density level segmentation task is also generated from the density map. For pixel p in input image i, its density level class S p,i is given as: where L represents the overall levels of density; following previous patch-based density level classification methods [7], [9], we set L equal to 4 in our work. D i is the pixel value in the i th density map ground truth.

B. Task Adaptive Backbone Network
Instead of using a shared backbone network to extract generalizable features for different tasks, we proposed an attentionbased task-adaptive shared backbone network to allow the model to extract discriminative features for the auxiliary tasks, thus helping to improve the performance of the main task. Fig.  2 shows the detailed structure of the proposed network, which consists of a shared backbone and three attention-based taskadaptive branches. To make a fair comparison with previous auxiliary task-based methods, such as [54], [14], [61], [9], etc., the truncated VGG-16 [77] is used as the backbone network. However, it can be replaced by any other robust network structure; we have reported the counting performance with other powerful network backbones in TABLE. II. The shared backbone adopts the first 13 layers of the VGG-16 to extract multi-level features. To exploit the global contextual dependencies, we proposed a Feature Fuse Block (FFB), which aggregated and fused the outputs from posterior layers back to the preceding layers hierarchically and iteratively, with up-sampling, concatenation and convolution operations. This provides improvements in extracting the full spectrum of semantic and spatial information across different stages and resolutions. The up-sampling is performed by using a bilinear interpolation algorithm. The convolution operation aims to reduce and match the corresponding feature map channel size between different stages.
With the aggregating process from low-level features to high-level features, the task-adaptive attention module is applied in three different task branches; details of the attention module are shown in the bottom left of Fig. 2. Each attention module consists of a global average pooling (GAP) layer to capture global context through different feature map channels, conducting an attention tensor to lead the emphasis of feature learning. Then, two blocks with a convolutional layer followed by a Batch Normalization (BN) [78] layer with ReLu and sigmoid as the activation functions are added. For the convolutional layer filter, the kernel size is 1 × 1. The element-wise multiplication is then performed between the outputs of the particular layer of the shared backbone and the task-specific attention module, which filters out the unrelated and redundant features from the backbone with respect to different auxiliary tasks and the main task. Therefore, the shared backbone can learn a generalizable representation, while the attention-based branches can extract task-specific features simultaneously in an end-to-end manner. The ablation study experiments proved that the attention-based adaptive backbone could boost the counting performance.
Apart from the aforementioned network structure component in three attention-based task-adaptive branches, we also introduced a cross-domain feature fusing operator in a particular order to focus on optimizing the primary density map regression task primarily. Specifically, the crowd seg-mentation branch is applied to the shared backbone first to select the corresponding discriminative spatial features. Then, we applied the density level segmentation branch on the shared backbone and crowd segmentation branch, which can enhance the additional contextual density level information into the main task. At last, the main task of the density map regression branch is applied.

C. Auxiliary Tasks
With three outputs from the task adaptive backbone network, we built two auxiliary tasks and a main task: crowd segmentation task, density level segmentation task, and density map regression task. We detail each of them subsequently. Crowd Segmentation. We introduced crowd segmentation as one of the auxiliary tasks for two reasons. Firstly, the density map's pixel value should be zero at non-crowd regions. However, the predicted density map can be noisy and inaccurate when the background is cluttered and complex. The crowd segmentation task provides a spatial focus to the density map regression process through zero out the non-crowd regions' pixel values. Secondly, given the standard set-up of single density map regression, the pixels within a particular range of the point annotations should contribute more to the final counting results; however, the loss is dominated by the majority of less relevant pixels. To overcome this limitation, the crowd segmentation can provide additional information enhancement in terms of the spatial indicator with a standalone loss function.
Given an input image I i ∈ R 3×H×W , we can get the output of the crowd segmentation branch in the backbone network, f CS ∈ R C×H×W , where H and W represent the height and width of the feature map; C is the channel size. Then, we apply a convolution layer with filter parameters θ CS ∈ R 1×1×1 , followed by a sigmoid as the activation function. Through this operation, we can generate a probability map to calculate the crowd and background probability. The single channel crowd segmentation probability map M CS is defined as: Density Level Segmentation. Density map regression is a pixel-wise task, which focuses on low-level features learning but may ignore high-level contextual information during the training [31]. To address this issue, we perform density level segmentation as another auxiliary task. Compared with previous patch-based density level classification methods [7], [54], [55], [9], our proposed pixel-based density level segmentation can provide pixel-wise level density information and highlevel semantic features at the same time. Upon the output of the density level segmentation branch of the backbone network f DS ∈ R C×H×W , a convolution layer with filter parameters θ DS ∈ R L×1×1 and a softmax as activation function are applied. The prediction of density level segmentation branch M DS is defined as: M DS = sof tmax(θ DS , f DS ) ∈ R L×H×W , where L is the number of density levels.

D. Density Map Regression
Intuitively, the different granularity features of density levels and spatial crowd locations need to be further reasoned to fuse into the density map regression branch. To this end, with the predicted crowd segmentation output M CS and density level segmentation output M DS as the auxiliary information granularity, we input them along with the density map branch's feature map f DM ∈ R C×H×W into the GCN reasoning module to reason the relationship among themselves. Subsequently, the output feature map f DM ∈ R C×H×W of the GCN reasoning module is reduced into one-channel through 1 × 1 convolution layer with a ReLU activation function.

E. GCN Reasoning Module
The proposed GCN reasoning module structure is shown in Fig. 3. In detail, there are three primary modules: Spatial Location Aware module, Density Level Aware Projection module, Graph Convolution on Vertices module. Because of the nature of the crowd images, the density level varies across the image [19], which indicates that the pixel values of the density map should not just rely on their own pixel-wise features but also on different density level regions. To this end, our GCN reason model projected a collection of spatial-aware density feature map's pixels with similar density levels to each graph vertex and exploited a GCN to reason about the relations among graph vertices. Spatial Location Aware Module. Before projecting the density map feature map f DM into the graph vertices, we directly applied the broadcasting Hadamard Product operation between the crowd segmentation output M CS and the density map regression branch's feature map f DM . There are two underlying reasons: (1) M CS is a one-channel crowd segmentation map, with encoding the probability of the non-crowd regions' pixel values approaching zero and crowd regions' pixel values approaching one; one can serve as a filter to zero out the non-crowd region's pixel value of the density map. (2) Direct broadcasting Hadamard Product can achieve crowd spatial awareness for every channel of the f DM through zero out the non-crowd region's pixel value. This can address the challenge of the complex scene backgrounds in the crowd images. Density Level Aware Projection Module. As mentioned above, the pixel-wise density level information can help to address the challenges of the large variations of density levels in crowd images. However, direct broadcasting Hadamard product between the density map branch's feature map f DM and the density level output M DS may result in domain conflicts [14]. We exploited the nature of GCN and projected the density level information into the graph vertices for further reasoning; one benefits the long-range relationship reasoning ability of GCN and the multi-granularity information enhancement from density level. Inspired by the non-local module [79], we encoded the long-range density level dependency among every pixel. Give the feature map M DS , the density level dependency matrix D D ∈ R HW ×HW is defined as: where Conv β and Conv are two convolution layers with 1 × 1 kernel size, respectively. The dependency matrix D D can be regarded as a pixel-wise attention map, where pixels with similar density levels are assigned with larger weights. The dependency matrix itself can reflect the pixel-wise density level dependency. Besides, we projected it as a prior to the graph domain through matrix multiplication, which can enhance high-level contextual dependency simultaneously. Graph Convolution on Vertices. In this module, we learnt how to reason the region-based relationship in density map through GCN in graph domain. Firstly, we projected the spatial aware feature map of f DM into graph domain with K vertices, and each vertex was represented by an embedding of shape H × W . This is achieved by Conv (µ), which is a 1 × 1 convolution layer. Furthermore, we projected the dependency matrix D D to the graph domain through matrix multiplication, resulting in the vertex features V D ∈ R K×HW . The projection aggregated pixels with similar density levels to graph vertices, where each vertex represents a region in the crowd image. Formally, V D is defined as: where ⊗ is matrix multiplication; is broadcasting Hadamard product. With the constructed vertices, the long-range region-  wise relationship is further reasoned in the graph domain through GCN. In detail, we reasoned over the region-wise relations by propagating information across vertices with a single layer GCN. Specifically, we fed the constructed vertex features V D into a first-order approximation of spectral graph convolution [80], resulting the output vertex features V D ∈ R K×HW . The V D is calculated as: where I is the identity matrix; A ∈ R HW ×HW denotes the adjacent matrix that encodes the graph connectivity to learn; W D ∈ R K×K is the weights of the GCN. The adjacent matrix A is randomly initialized but can learn and update the edge weights from vertex features by gradient along the training process. The identity matrix I serves as a residual connection that alleviates the optimization difficulties. Based on the learned graph, the information propagation across all vertices leads to the finally reasoned relations between regions. After graph reasoning, a collection of pixels embedded within one vertex share the same context of features modeled by graph convolution. Then, we re-projected the vertex features in the graph domain to the original pixel grids. Given the reasoned vertices V D , we applied Conv (σ), which is a 1 × 1 convolution layer. Finally, we summed up the re-projected refined and the original density feature maps as the final feature map. The final pixel-wise density feature map f DM is thus computed by

F. Loss Function
The whole network is end-to-end trainable, which includes four loss functions; the total loss function is defined as: where γ is empirically set as 2, which is a hyper-parameter to trade-off between auxiliary losses and main loss. Please note that, extensive experiments have been done to determine the weights of the losses for two auxiliary tasks, respectively. We found that there is no significant difference of counting performance with respect to different weight values; thus, we set them both equal to 1 in the loss function. Binary cross-entropy (L CS ) is used for crowd segmentation auxiliary task; categorical cross-entropy (L DS ) is used for density level segmentation auxiliary task; L2 loss is used for pixel-wise density map regression supervision (L Dp ). However, pixelwise L2 loss assumes pixel-wise independence, which results in over-smooth density map prediction [8] and the underlying bias from unbalanced low-and high-level density distributions of crowd images. To address the issue, we proposed Dilated Contrastive Density Loss (L DCD ), where we take into account more adjacent pixels for regional density difference. In detail, we applied single layer convolution on the regressed density map M D and the ground truth density map D G respectively. The single layer convolution has eight filters; each filter contains dilated kernel with a fixed value (e.g. 1, 0, and -1). The least-square error of the calculated regional dilated contrastive values from the regressed and ground truth density map is the output of L DCD . To this end, we define L DCD as below: where K DCD i is the i th dilated contrastive convolution kernel, i ∈ [1,8]. Details of the kernel are shown in Fig. 4, where a 3 × 3 convolution layer with the dilated rate of 2 is applied; one gives a larger receptive field as 5 × 5. We perform extensive experiments to evaluate the effectiveness of the proposed L DCD loss; quantitative results in Ablation Study (Section V-C) demonstrates that the proposed L DCD loss can improve the counting accuracy not only for our model but also for previous single L2 loss based methods.

A. Datasets
ShanghaiTech [18] consists of 1,198 images, containing a total amount of 330,165 people with head centre point annotations. This dataset is divided into two parts: SHA includes 482 images, in which crowds are mostly dense (33 to 3139 people); SHB includes 716 images, where crowds are sparser (9 to 578 people). Each part is divided into training and testing subset as specified in [18]. UCF-QNRF [82] is a large crowd dataset, consisting of 1,535 images with about 1.25 million annotations in total. The number of people in these images varies largely with a wide range from 49 to 12,865. As indicated by [82], For training, 1,201 images are used, the remaining 334 images form the test set. JHU-Crowd [83] is a recent challenging large-scale dataset that containing 4,372 images with 1.51 million annotations. The dataset includes several challenging scenes such as weather-based degradation and illumination variations etc.. This dataset is divided into 2,272 images for training, 500 images for validation, and 1,600 images for testing. NWPU-Crowd [22] is up to date the largest public crowd counting dataset, containing 5,109 images with over 2.13 million annotations. The dataset includes 3,109 training images, 500 validation images and 1,500 test images. Moreover, inspired by the potential of crowd counting, we conducted experiments on commonly used cell counting dataset: DCC [84] with 100 images for training and 77 images for testing, and vehicle counting dataset: Trancos [85] with 403 images for training, 420 images for validation and 421 images for testing. These experiments further demonstrated our model's robustness and applicability for different realworld applications.
Note that, for ShanghaiTech (SHA, SHB), UCF-QNRF, and DCC dataset, we use 10% of the given training images as the validation dataset.

B. Implementation Details
To augment the dataset, we randomly cropped the input images, density maps, crowd segmentation masks, and density level segmentation masks with fixed size 128×128 at a random location, then randomly horizontally flipped the image patches with the probability of 0.3. We trained our model for 400 epochs for all experiments, with a start learning rate of 1e − 4 and a cosine decay schedule [86]. The batch size is set to 96. All the training processes are performed on a server with 8 TESLA V100 and 4 TESLA P100, and all the test experiments are conducted on a local workstation with a Geforce RTX 2080Ti. Five-fold cross-validation is used for fair comparison and hyper-parameters tuning in all settings.

C. Evaluation Metrics
To evaluate the counting performance, we adopted Mean Absolute Error (M AE) and Root Mean Squared Error (RM SE). Since Mean Absolute Error (M AE) and Root Mean Square Error (RM SE) cannot measure the counted objects' locations, Grid Average Mean absolute Error (GAM E) is used to indicate counting accuracy over local regions. GAM E is defined as: where N is the total number of images, y l n andŷ l n are the ground truth and estimated counts in the local region l of n th image. 4 L denotes the number of non-overlapping regions which cover the full image. When L equals to 0, the GAM E is equivalent to M AE.

A. Counting Results
In this section, we present our experimental results on the crowd, cell, and vehicle counting tasks in comparison to other auxiliary-task based state-of-the-art crowd counting methods. These experiments further demonstrate our model's robustness and applicability in multiple domain datasets. In the Discussion (Section V-D), we showed that our model could indicate some mislabeled or wrongly labeled point annotations from the ground truth of the test dataset. This highlights our approach's generalizability and the potential issue of imperfect ground truth in object counting datasets. Crowd Counting Results. We performed experiments to validate our model's performance in five challenging crowd  counting datasets. Fig. 5 shows qualitative results; specifically, we presented the predictions from auxiliary task branches (crowd segmentation and density level segmentation masks) to demonstrate our model's cohesion, along with the spatial location and density level variation's contribution of auxiliary branches. To make a fair comparison, we only compared our model with previous auxiliary task learning based counting methods.  [84]) and vehicle (Trancos [85]) counting datasets to show our model's broad applicability and robustness. Fig. 7 shows the qualitative results, and Fig. III shows the quantitative results compared with the previous state-of-theart methods. Due to the different scenes in the cell counting dataset, such as less occlusion, no scale variation, no complex background etc., the contribution of some components of our model will be lessened because we design our model especially for crowd counting tasks; still, our model achieves comparable performance with previous methods. Furthermore, we presented local comparison performance through the GAM E metric to indicate the model's ability to recognize the objects' locations. Fig. 6 shows the comparison results in terms of the GAM E on the Trancos dataset. As illustrated, our method   localizes and counts overlapping vehicles more accurately.

B. Auxiliary Task Results
In this section, we reported the performance of two auxiliary tasks. The commonly used segmentation metric Intersection over Union (IoU) is used to evaluate the auxiliary tasks' performance. In detail, we achieved average 88.7 % IoU for the crowd segmentation task and 81.0 % IoU for the density level segmentation task on the five crowd counting datasets.

C. Ablation Study
We investigated the effect of each component in our proposed model. All ablation experiments were performed with the same settings detailed in the Implementation Details (Section IV-B). Ablation on Different Network Backbones We evaluated the effectiveness of different backbone networks on the five crowd counting datasets. The counting performance is shown in TABLE. II with several different backbone networks. In general, VGG-based backbone networks achieved comparable counting performance, compared with the one of ResNet-based backbone networks in relatively large-scale datasets, such as QNRF, JHU-Crowd and NWPU-Crowd. While, ResNet-based backbone works better on small-scale counting datasets, such as SHA and SHB. We reported our model's performance with VGG-16 backbone network in TABLE. I for a fair comparison with previous methods. Ablation on Auxiliary Tasks and Model Components. In this section, we evaluated the effectiveness of the auxiliary tasks, adaptively shared backbone network, and GCN-enabled reasoning module, respectively. Please note that, in order to eliminate the performance improvement from a bigger model, we add feed-forward CNN blocks (3 × 3 convolution with Batch Normalization) into other ablation study models in TA-BLE. IV to maintain a similar model size as ours (18.8 million parameters). Firstly, we compared the single task density map regression network, in which we removed the GCN reasoning module, the auxiliary learning branches, and the adaptively shared backbone branches, to form a single column network structure (Single Column). Then we added two auxiliary branches separately and simultaneously after the single shared backbone's output to form an auxiliary learning mechanism (w/ Crowd Seg, w/ Density Seg, w/ Both Auxiliary). To further improve the performance, we designed and added an adaptive backbone network to enable the task-shared and task-specific features being learned simultaneously (w/ Adaptive Crowd Seg, w/ Adaptive Density Seg, w/ Both Adaptive Auxiliary). Furthermore, we evaluated the proposed GCN reasoning module's effectiveness, which can propagate region-based density level information across the image (Ours). The effect of each structural component is presented in Fig. IV. As illustrated, the proposed auxiliary task learning mechanism (w/ Both Auxiliary) is reduced by 14.3% over the single-task learning method (Single Column) via average MAE on two datasets, the task adaptive backbone (w/ Both Adaptive Auxiliary) reduces 6.8% over the single shared backbone (w/ Both Auxiliary), and the GCN reasoning module further reduces 6.7%. Qualitative comparison results of different modules' effectiveness in terms of predicted density maps are shown in the Fig. 8, where the crowd segmentation auxiliary (w/ Adaptive Crowd Seg) can help the model to focus on the features in the region of interest and filter out the background (first and second rows).
On the other hand, the density level segmentation auxiliary (w/ Adaptive Density Seg) can help to estimate more accurate density levels across the whole density map (second and third rows). We highlighted the different areas among those ablated models' density map predictions with red bounding boxes for better visualization and comparison. Moreover, in TABLE. V, we further indirectly evaluate the auxiliary tasks' effectiveness in this work. Specifically, for other ablation study models except for Ours, we maintained the same network structure as Ours to keep the same model size (18.8 million parameters) but switched off the two auxiliary tasks' loss functions. In TABLE. V, it proves that the supervision from multi-granularity information of auxiliary tasks contributes to the final counting performance in this work. Without L CS and L DS losses, the counting error increases by an average of 21.75 % on the SHA and the JHU-Crowd datasets via MAE. Ablation on Loss Function. We performed experiments to evaluate the receptive field through different dilated rates in the proposed dilated contrastive density loss function L DCD . In detail, we changed the dilated rate of the 3 × 3 convolution layer into 1, 2, 3, 4, which resulted in the receptive field of the L DCD being like 3, 5, 7, 9.  performance on SHA and JHU-Crowd datasets. Furthermore, we conducted experiments to evaluate the effectiveness of the proposed dilated contrastive loss function, in which we removed the L DCD and kept the rest of the network constant with the same trade-off hyper-parameters (Base in TABLE. VII). Furthermore, we applied the proposed combined loss function (w/ contrastive in TABLE. VII) into previous single L2 based methods. We re-implemented their network with their open-source code and used the same experimental setting as our method. Fig. VII shows the comparison results of our proposed combined loss function; as illustrated, with regional density difference supervision of L DCD , our model attains a 3.5% reduction compared with single L2 loss function via average MAE on two datasets. Our proposed L DCD also helps to reduce the original MCNN [18] by 6.4%, the CSRNet [33] by 2.7%, and the CACC [19] by 2.3% over average MAE on two datasets. Please note that, we did not compare with other loss functions that were proposed in recent crowd counting model [69], [71], [70], [73], [74], [72]. Because those methods are not pure density map regression based methods, it is unfair to compare.

Input
Prediction Ground truth Input Prediction Ground truth

D. Discussion: Comparison with Ground Truth
The underlying labeling errors (noisy ground truth) exist in most datasets due to the human annotators' errors. However, a robust model can omit the noise ground truth during training and produce a more accurate prediction. This section showed that our model could indicate some mislabeled or wrongly labelled point annotations of the ground truth in the test dataset. This highlights the generalizability of our approach and the potential issue of the imperfect ground truth in object counting applications. Fig. 9 shows a wrongly labelled point annotation (top left) case of the crowd counting test dataset, and the other cases are mislabeled point annotation of vehicle and cell counting test dataset. We highlighted the wrongly labelled or mislabeled area with red bounding boxes for better visualization and comparison.

VI. CONCLUSION
We proposed a novel framework for auxiliary task learning based counting by employing an adaptively shared backbone, a GCN reasoning module and a novel dilated contrastive density loss function. Our model advocates task-shared and task-specified features to be learned simultaneously. The proposed method highlights that cross-domain reasoning in graph through GCNs using crowd segmentation and density level segmentation can significantly improve feature learning in density map regression tasks. With our proposed loss function's regional density difference supervision, our model set a new state-of-the-art among auxiliary task learning based counting methods on seven challenging benchmarks.