{'fraction_non_alphanumeric': 0.036081978254594436, 'fraction_numerical': 0.018955065909746945, 'mean_word_length': 4.770868036276143, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Ambient Sound Helps: Audiovisual Crowd Counting in Extreme Conditions

Visual crowd counting has been recently studied as a way to enable people counting in crowd scenes from images. Albeit successful, vision-based crowd counting approaches could fail to capture informative features in extreme conditions, e.g., imaging at night and occlusion. In this work, we introduce a novel task of audiovisual crowd counting, in which visual and auditory information are integrated for counting purposes. We collect a large-scale benchmark, named auDiovISual Crowd cOunting (DISCO) dataset, consisting of 1,935 images and the corresponding audio clips, and 170,270 annotated instances. In order to fuse the two modalities, we make use of a linear feature-wise fusion module that carries out an affine transformation on visual and auditory features. Finally, we conduct extensive experiments using the proposed dataset and approach. Experimental results show that introducing auditory information can benefit crowd counting under different illumination, noise, and occlusion conditions. The dataset and code will be released. Code and data have been made available


Introduction
Crowd counting has recently been a hot research topic [2,26,38,37], as it can benefit a wide range of applications, to name a few, safety monitoring, public space design, and disaster management. Consequently, crowd counting techniques, particularly computer vision-based approaches, have received increased interest. The success of current state-of-the-art visual crowd counting models can be attributed to the development of convolutional neural network (CNN) architectures that aim at learning better visual representations from images for this task [22,25]. Albeit successful, vision-based crowd counting approaches could fail to capture informative features in extreme conditions 6 (c.f., Fig. 1).
The first three authors contribute equally to this work. 6 In this paper, the extreme condition refers to a) low resolution, b) noise, c) occlusion, and d) low illumination.  Fig. 1: Crowd counting on low-quality images. From left to right: input image with low illumination and strong noise, ground truth density map, predicted density map using both auditory and visual information and predicted density map only using visual information.
Investigations in the field of neurobiology show that human perception usually benefits from the integration of both visual and auditory information [42], e.g., lip reading, where correlations between lip movements and speech provide a strong cue for linguistic understanding [3]. This gives us an incentive that ambient sound could be an important cue for identifying the number of people in a scene. This hypothesis is in line with our daily experiences: the louder we perceive the ambient sound to be, the more people there are. However, incorporating the ambient sound into a visual crowd counting model and its contributions to this task still remain underexplored in the community. On the other hand, with the now widespread availability of smartphones, digital cameras, and video surveillance equipments, audiovisual data have been accessible at a reasonable cost. This enables us to explore the topic in this paper.
In this paper, we are interested in a novel task, audiovisual crowd counting. We pose and seek to answer the following questions: -Is combining features coming from visual and auditory modalities better than only using visual features for crowd counting in extreme conditions? -How do audiovisual crowd counting results vary under different illumination, noise, and occlusion conditions? -How do we impose the audio information for effectively assisting the visual perception, i.e., how to fuse both modalities?
To this end, we establish an annotated benchmark, auDiovISual Crowd cOunting (DISCO) dataset. This allows for quantitative comparisons. As to the fusion of two modalities, inspired by FiLM [32], we also employ a feature fusion block where audio and visual features are fused in a linear manner. The advantages of linear feature fusion is its simplicity and less consumption of time and computational resources. To better fuse the two modalities, we stack multiple feature fusion blocks in our implementation (details are shown in Section 4). This work's contributions are threefold.
-Inspired by the audiovisual perception ability of human-beings, we investigate a novel audiovisual crowd counting task. We are not aware of any previous work exploring this topic.
- To facilitate progress in this field, we provide a large-scale benchmark, which  consists of 1, 935 pairwise image-audio clips and 170, 270 annotated

Related Work
Crowd Counting. Early approaches [6,15,21,35,40,22,45,25] mainly focus on estimating the number of people in crowd scenes with hand-crafted features, including Harr-like [44], HOG [7] and so on. Recently, CNN-based methods [4,51,36,16,17] have attained remarkable improvements by introducing architectures, such as multi-column, multi-stream, and trellis structure. Among these studies, [22,25,40,45] aims at extracting global features or expanding the size of respective fields to encode the contextual information comprehensively. To fully mine the power of CNN, some methods propose novel feature fusion strategies (simple concatenation, element-wise sum operation [24], adversarial learning [40], iterative CNN [33], bottom-top/top-bottom fusion [41]) to integrate multiple deep features from different layers or modules. Although CNN-based approaches achieve a significant progress, there are two issues in the traditional single-image counting: 1) object occlusion results in missing estimation; 2) RGB sensors are susceptible to light intensity, object occlusion, visibility, etc. To address the first problem, some researchers [50,28] propose multi-view methods to count the number of people. However, it is not easy to simultaneously carry multiple cameras with specific parameters and camera calibration in the real world. Regarding the second problem, although Lian et al. [23] attempt to solve it by introducing depth information, it is infeasible to tackle above issues at the same time. Joint Audiovisual Representation Learning. Joint audiovisual learning is expected to reward the learning model merits from both visual and auditory modalities. Early audiovisual researches mainly focus on speech recognition [8] as the visual message is considered to be free of audio noise and can provide complementary information in the noisy condition. A certain improvement usually can be achieved by jointly modeling the facial/mouth movements and corresponding audio signals [30,12]. Similar phenomenons can also be found in other recognition tasks, such as affect recognition [48] and gesture recognition [18]. Recently, audiovisual learning has been further employed to analyze more general scenarios. Owens et al. [31] propose to transfer knowledge learned from audio modality to supervise the training of visual recognition models. Furthermore, Arandjelovic et al. [1] propose to analyze video scenes with only audiovisual correlations, which could then be applied to sound localization [13,14] and separation [9]. Inspired by these audiovisual works, we propose to transmit the merits of audio modality for crowd counting, which is considered to be free of visual noise and could provide considerable references.

Dataset
To jointly utilize ambient sounds and visual contexts for crowd counting, an auDIoviSual CrOwd dataset ("DISCO" for short) is constructed. In this section, we will describe the proposed dataset from the following two perspectives: data collection and data characteristics.
Data Collection. To simultaneously capture the visual image sequences and record the audio signals, we use four video cameras, HDR-CX900E produced by Sony Corporation. In the collection process, we simulate the view of a surveillance camera and record crowds in some typical scenes at different time. As a result, we collect 248 video clips, around 20 hours and 385 GB data in total. Specifically, the resolution of each video is 1, 920 × 1, 080, and the frame rate is 25. For the audio information, the DV record 2-channel stereo with the sample rate of 48, 000.
From these raw data, 1,935 images and audios from various typical scenes are selected to construct our proposed dataset. For an image at t in a video, we extract its corresponding audio signals from t − 0.5s to t + 0.5s. Some visual examples and their corresponding audio waveforms are shown in Fig. 2.
Data Characteristics. DISCO consists of 1, 935 crowd images, a total of 170, 270 instances annotated with the head locations. The average, minimum and maximum number of people for each image are 87.99, 1 and 709, respectively. Fig. 3 reports the histogram of the population distribution. From this, we find that the number of people in most scenes is between 0 and 100.
Compared with some traditional crowd counting datasets [51,16], the proposed DISCO dataset are the first to record ambient sounds as auxiliary information of crowd scenes to reduce defects of single-vision sensors. In addition, we 0 -1 9 2 0 -3 9 4 0 -5 9 6 0 -7 9 8 0 -9 9 1 0 0 -1 1 9 1 2 0 -1 3 9 1 4 0 -1 5 9 1 6 0 - capture images at different times in one day to ensure their various illuminations (see Fig. 2). As illustrated in Fig. 3, We also analyze the illumination distribution in YUV space of DISCO and compare it with Shanghai Tech Part A/B (SHT A, SHT B for short) [51] and UCF-QNRF [16]. It can be observed that the illumination of 25%/3% of images in DISCO dataset is extremely low/high (see the first two and 9th bins), while other datasets rarely involve very few images of such poor quality. What's more, different from traditional video-surveillancestyle datasets [5,6,49], the proposed DISCO dataset covers various scenes from different cities, such as subway station, mall, restaurant, campus, plaza, stadium, sidewalk, etc. In a summary, DISCO dataset has three advantages comparing with others: 1) both audio and visual signals are provided; 2) cover different illuminations; and 3) a large variety of scenes are considered.

Overview
In order to benefit crowd counting with ambient sounds, a novel AudioVisual Counting (AVC) network is designed and consists of three modules (see Fig. 4): (1) visual feature extraction, (2) audio feature extraction, (3) audiovisual feature fusion. Notably, comparing with traditional methods [4,16,17,51,36], where only visual information is employed, our network is characterized by the second and third module. As aforementioned, one of our motivations is that in the scenario of low-quality images, such as low resolution, low illumination and strong noise, it could be difficult to precisely count people with only visual information. In contrast, humans is able to estimate the number of people, even though we cannot see them, the reason is that we are able to perceive the scene by hearing. To imitate such human capacity, we introduce an audio module into the traditional counting framework, resulting in AudioVisual Counting (AVC) model.

Visual Feature Extraction
Similar to CSRNet [22], we employ the first ten layers of VGG16 [39] as the front-end CNN V CN N to extract visual features. Given an RBG image I with a spatial size of W × H, visual features v f eat can be extracted with the following equation where v f eat ∈ R C× W 8 × H 8 , and C denotes the number of channels, i.e., 512.

Audio Feature Extraction
In this work, we use Log Mel-Spectrogram (LMS) for representing audio and CNN arch for modeling due to following considerations: 1) The audio feature of LMS has been widely used in CNN-like neural model for sound event detection and shown noticeable performance [10], and 2) Stoter et al. [43] demonstrates that using spectrogram-like feature can achieve comparable performance to the conventional MFCC in the counting task and much simpler. Even so, we still provide some discussions about different audio features and modeling settings in the experiments. Given a raw audio signal A raw = {a 1 , a 2 , · · · , a T }, we first sub-sample A raw at 16kHz, and then employ short-time Fourier transform (STFT) using Hann window with the window size of 400 and a hop length of 160, to generate a 98 × 257 time-frequency map. Afterwards, Mel filter bank is applied, and a 96 × 64 representation A spec can be then obtained for each raw audio signal. Finally, we utilize a VGG-like deep convolutional neural network [10] to extract audio features a f eat as follows: where a f eat ∈ R C×Wa×Ha , and C = 512.

Feature-wise Audiovisual Fusion
To effectively fuse both audio and visual information in crowd counting, we introduce a feature-wise fusion module which aims at adaptively adjusting visual feature responses with transformed audio embeddings. Concretely, based on the extracted audio features, two feature-wise parameters γ and β are learned to model such cross-modal influence in terms of multiplicative and additive aspects, respectively. The formula is shown here: where v l ∈ R C l ×W l ×H l indicates outputs of the lth feature fusion block, D l CN N denotes the lth dilated convolution layer, F l and suggest the activation function and element-wise multiplication, respectively. Notably, l ranges from 0 to 6, and v 0 = v f eat . Normally, γ and β can be learned via different affine transformations, such as single or multiple neural networks. In this work, we simply use fully-connected layers to learn γ l and β l with the following two equations: In these two equations, AvgP represents average pooling, and γ, β ∈ R C l+1 . To implement Eq.3, γ l and β l are tiled to match the size of visual features before fusion, see Fig.4. Intuitively, the feature-wise fusion module manipulates visual feature maps by referring to their corresponding ambient sound information independently, especially when faced with low-quality images. Note that, as γ and β are irrelevant to the spatial dimension of visual features, the fusion module can be applied after arbitrary visual feature maps, which makes it possible to further improve the fusion performance by inserting the fusion module at different levels simultaneously. In this condition, we can also share the same affine transformation in different fusion modules for an efficiency purpose, as shown in Fig.4.

Loss Function
Given a ground-truth density map Y ∈ R W ×H and a predicted density map Y ∈ R W ×H , we select L 2 norm as the loss function, and the loss can be calculated with the following equation:

Experiments
In this section we present the settings, results, and analysis of the experiments.
More results and discussions are in the Appendix.

Experimental Settings
First, we split our DISCO dataset into three sets: 200 images for validation, 300 images for testing, and the remaining 1,435 images for training. To obtain the ground-truth density maps, we convolve each binary annotations (centers of human heads are one, and the others are zero) with a 15 × 15 Gaussian kernel K ∼ N (0, 4.0). In the training phase, we select Adam [19] as the optimizer and set its parameters as recommended. The learning rate is initialized as 1e − 5 and decays by 0.99 every epoch. To alleviate overfitting, weight decay is employed with a λ of 1e-4. It is noteworthy that except for those with a low resolution of 128 × 72, we resize images into 1024 × 576 to reduce computational resources and time.
In our experiments, the batch size is set as 4, and the maximum training epoch is 500. At the end of each epoch, models are evaluated on the validation split, and only those of the best performances are remained after training. To fairly compare all models, we report their performances on the test split.
Regarding the audio CNN, we use VGGish [10] pre-trained on audioSet [20] and discard its last three fully-connected layers, resulting in a 6-layer CNN. For the visual front-end CNN, we use the first ten layers of VGG16 [39] pre-trained on ImageNet [34].
In our feature fusion blocks, we employ dilated convolutions [47] with the kernel size of 3 and the dilation rate of 2 to enlarge the receptive field. Similar to CSRNet [22], we stack 6 fusion blocks and up-sample outputs with a factor of 8 to yield density maps of full resolution.

Baselines and Evaluation Metrics
To investigate the task of audiovisual crowd counting, we compare our audiovisual counting model with several vision-based models, such as MCNN [51], CSRNet [22], SANet [4], and CANNet [25]. Notably, we use one of the stateof-the-art models, CSRNet [22], as the backbone of our audiovisual counting model, leading to the proposed AudioCSRNet. The architecture of our proposed network is shown in Fig. 4.
To assess the performance of each model, we employ Mean Absolute Error (MAE) and Mean Square Error (MSE) scores, which are computed as follows: where N denotes the number of images, c andĉ denote the ground-truth and predicted numbers of people.

Audiovisual Crowd Counting on Low-quality Images
To evaluate the performance of our AudioCSRNet in crowd counting under extreme conditions, we conduct experiments on two extreme scenarios: 1) the quality of images is very low, and 2) occlusion exists in images. In this subsection, we focus on the first scenario and test our model on three low-quality conditions: low illumination, low resolution, and strong noise. Specifically, we mimic images taken in the dark environment 7 with the method proposed by [27]. First, we randomly reduce the brightness of images with a rate of r, calculated by R * Uniform(0, 1). Then we add Gaussian noise to the low illumination image and the standard deviation σ of the Gaussian noise is a random variable, 7 The problem of directly using natural extreme low-illumination images is that the annotation of human heads in these images is difficult owing to insufficient viual information. To solve this, we first label images with low illumination and then produce pseudo extreme low-illumination by imitating the extreme scenario. i.e., σ = Uniform(0, 1) * B 255 2 , where R and B are two hyper-parameters to control brightness decay rate and the variance of Gaussian noise. To quantitatively measure the quality of input images, here we calculate PSNR [11] and BRISQUE [29]. Notably, images with high PSNR and BRISQUE scores are regarded as high-quality ones. 8 . Table 1 shows comparisons among different models. Specifically, comparisons between AudioCSRNet and its counterpart, CSRNet, directly demonstrate that introducing audio information can benefit crowd counting, in particular on lowerquality images. For example, on images with low illumination and Gaussian noise (PSNR is 9. Another advantage of introducing audio into crowd counting is that audiovisual models show strong robustness on variant scenarios, e.g., AudioCSRNet obtains the lowest average MAE and MSE score (19.21 and 36.86). According to the no free lunch theorem [46]-no model is able to outperform the others in all scenarios, while introducing audio to crowd counting balances the performance and in most cases audiovisual models performs better than models using only visual information.
To further evaluate the effectiveness of AudioCSRNet, we conduct ablation studies with different illumination reduction strategies, as shown in Fig. 5. Generally, AudioCSRNet is able to obtain lower MAE and MSE score when the illumination is reduced gradually. An interesting observation is that when R is 0, which indicates that nothing can be observed, AudioCSRNet still works and can achieve an MAE of 60.43 and MSE of 89.45. This means that even with only ambient sounds, AudioCSRNet can count people. This is in line with the reality that human can roughly estimate the number of people with only auditory information. Although the number estimated by hearing is not precise, low-quality images can be used to calibrate it. For instance, when R = 0.2, B = 25, MAE and MSE obtained by AudioCSRNet decrease to 25.06 and 51.58, respectively, which is much lower than those obtained by CSRNet (35.78 and 62.76). By increasing the intensity of Gaussian noise, such difference enlarges as well, i.e., the MAE of AudioCSRNet is decreased by 18.55 compared to that of CSRNet, when R = 0.2, B = 50.
Also, we show the density map predicted by CSRNet and AudioCSRNet in Fig. 6. Jointly applying auditory and visual information leads to more accurate density maps, especially for images that suffers from stronger noise and lower illumination. In Fig. 6, the first row shows the comparison on a high quality image, where density maps predicted by AudioCSRNet and CSRNet are quite similar, while in the last row, the prediction of AudioCSRNet is superior to that of CSRNet when image quality decreases. To summarize, introducing audio information can effectively improve the performance of crowd counting with lowquality images

Audiovisual Crowd Counting under Occlusion
Another scenario that we study is occlusion, where an input image is randomly occluded by a black rectangle. Specifically, given an image I with the size of W × H and an occlusion rate O r , we first generate a black rectangle with the size of int(W √ O r ) × int(H √ O r ), and then randomly mask out certain image contents with this black rectangle. Results are shown in Fig. 7, and we can see that performances of both CSR-Net and AudioCSRNet on occluded images dramatically decrease, e.g., CSRNet and AudioCSRNet achieve MAE scores of 23.20 and 17.47, respectively, when 20% of input image area is occluded, which is 1.8 and 1.3 larger than the MAEs obtained on fully visible images. Moreover, trends of curves in Fig. 7 demonstrate that AudioCSRNet can often achieve lower MAE and MSE scores, and with an increasing O r , the gap between MAE and MSE scores achieved by CSRNet and AudioCSRNet grows as well, meaning that the changes of images can more easily affect CSRNet that only depends on visual information, while AudioCSRNet is more stable when the quality of images varies.

Vision vs. Audiovision on High-quality Images
Despite the success of AudioCSRNet in crowd counting under extreme conditions, here we are interested in whether ambient sounds help when high-quality images are available as well. Table 2 shows comparisons between different models on high-quality images. Obviously, performances of all models on high-quality images increases compared to those on low-quality images (see table 1). As shown in Table 2, CSRNet outperforms all other models and achieves 13.88 in terms of MAE. Besides, CANNet performs worse than CSRNet, and gains increments of 1.53 and 0.17 in MAE and MSE, respectively. Moreover, we find that the performance of AudioCSRNet is comparable to that of CSRNet, which achieves slightly worse MAE score (14.24), but better MSE score (28.07), and this interesting observation demonstrates that the introduction of audio information might not hurt the performance on high-quality images. One possible explanation could be that high-quality images (BRISQUE score reaches 78.03) can provide sufficient visual cues for the precise estimation of crowd counts. Notably, although images in our dataset cover a wide range of the illumination (see Fig.  3 right), those with extremely low illumination are not used in the annotation procedure. As a consequence, people are recognizable in all collected images. In this case, the application of audio information might bring noise and slightly reduce the network performance.
By comparing experimental results on high-quality image with low resolution images (see table 1), it can be observed that the MAEs obtained by CSRNet and CANNet decrease by 19% and 30.5%, respectively. This is because CANNet  learns and aggregates features of different scales with multi-scale structures, and those extracted from low resolution images might introduce noise. An intuitive explanation could be that a person in a low resolution image consists of only a few pixels, and a kernel with the large receptive field might take irrelevant visual information, e.g., noise or background, into consideration when counting people.

Audio Net as a General Module
Note that our audio net shown in Fig. 4 can also be applied to any other visionbased models. In this subsection we directly applied the audio net to CANNet, resulting in AudioCANNet, where the backbone is the first 10 layers of VGG16 and the scale-aware layer. As shown in Table 3 28.96) on high-quality images, therefore, in the case of using high-quality images, audio could be still helpful or at least not significantly reduce the performance (see table 2), but introducing audio makes model more stable in different scenarios, such as lowering illumination or adding noise.

Discussion
Recall the questions that we want to answer (see Section 1), we discuss in this section.
First, combining visual and auditory information is able to benefit crowd counting, in particular in the scenarios of low illumination, strong noise, low resolution and occlusion. If the image quality is high, introducing audio could result in comparable performance, since the visual information is good enough for counting. Another advantage of jointly applying auditory and visual cues is that the robustness of models in different scenarios can be enhanced. Models that only rely on vision could fail in some extreme scenarios, by contrast, combining auditory and visual information is able to handle the extreme scenarios.
Second, when illumination degrades, the performance of vision-based models could dramatically decrease (see Fig. 5), whereas the audiovisual counting model is capable of obtaining lower MAE and MSE. Interestingly, the gap between vision-based and audiovisual models enlarges with the decrease of illumination and when there is no visual information, audiovisual counting models are able to estimate the number of people by only "hearing" the ambient sound. In terms of occlusion, audio is also helpful, in particular for the models that are sensitive to occlusion, such as CSRNet. And similar trend occurs in the scenario of occlusion-the gap enlarges with the increase of occlusion rate. Moreover, for low resolution images, combining audio and vision performs relatively well.
Third, in this paper we investigate in linear feature fusion approach, since it is simple and performs relatively well on other multi-modality tasks such as visual question answering (VQA) [32]. And for the combination of vision and audio, linear feature fusion still performs well and in most cases audiovisual counting models outperforms their counterparts that only use visual information. In the future, we will pay more attention to how to fuse audio and vision for crowd counting and scene understanding.
Last but not least, we investigate in using different audio representations in AudioCSRNet and the results are shown in table 4. Obviously, using CNNs to extract high-level features from spectrogram and MFCC is better than using LSTM. The possible reason is that CNNs consist of much more parameters than LSTMs, thus more powerful. Interestingly, MFCC could obtain similar performance compared with using spectrogram (see table 1), however, the computation of MFCC is slightly complex.

Conclusion and Outlook
In this paper, we investigated a novel audiovisual task, that imposes audio information for assisting visual crowd counting in extreme conditions. We developed an audiovisual crowd counting dataset to facilitate progress in this field, which covers different scenes in different illuminations. Meanwhile, a feature-wise fusion model was developed to achieve audiovisual perception for crowd counting. Extensive experiments were conducted to explore audio effects in different visual conditions. We found that introducing audio is able to benefit crowd counting, in particular in the extreme conditions, such as low illumination, strong noise, low resolution and occlusion.
In the future, three directions should be considered. First, what is the best way to fuse audio and vision? In this paper, we studied a simple fusion approach, and some other methods should be investigated in the future. Second, does audio benefit crowd counting in other conditions, such as cross-scene? Third, if audio and video for the same scene are collected in different positions, does audio still benefit crowd counting? Since we lower the illumination and add Gaussian noise to the images, a naive approach for crowd counting is to first enhance the corrupted images and then feed them into the counting models. In our experiments, we use R = 0.2 and B = 25, 50 to randomly corrupt the input images and then we employ a Gaussian filter with 11 × 11 kernel size and the standard deviation is adaptive to the size of kernel. After that we employ histogram equalization approach to alleviate the effect of lowering illumination. Table S5 shows the performance on enhanced images. Although the input images are enhanced, MAE and MSE scores are much higher than using highquality images, e.g., CSRNet obtains MAE of 12.73 and MSE of 26.99 (see table 1 in our paper) using high-quality images, while it obtains 43.06 and 67.11 for MAE and MSE using enhanced images. Compared with using corrupted images (see table 2 in our paper), CSRNet performs better using enhanced images, e.g., it obtains MAE and MSE of 33.83 and 55.59 using enhanced images, by contrast it obtains 35.78 for MAE and 62.76 for MSE using corrupted images with R = 0.2, B = 25. Also, when R = 0.2, B = 50, using enhanced images leads to better performance, e.g., 43.06 vs. 45.88 for MAE and 67.11 vs. 75.40 for MSE. Interestingly, in terms of AudioCSRNet, using enhanced images results in slightly worse performance than using corrupted images based on MAE, e.g., 25.25 vs. 25.06 and 33.00 vs. 27.33 for B = 25 and B = 50, respectively. The possible reason is that, after enhancing, people in images become more recognizable, in particular, the effect of low illumination is mitigated. Comparing AudioCSRNet to CSRNet, introducing audio significantly improves the performance, e.g., Au-dioCSRNet obtains 25 Input Image Spectrogram GT CSRNet AudioCSRNet Fig. S8: Generated density maps on occluded images. In the case of occlusion, it is difficult to predict an accurate density map, in particular for the occluded area. In most cases, introducing audio is able to predict a more accurate number of people (see the numbers in the figure), which indicates that audio may not help us to localize people to generate accurate density maps when the images are in extremely low quality, but it is able to benefit counting the number of people.

Appendix
Input Image Spectrogram GT CSRNet AudioCSRNet Fig. S9: Generated density maps on enhanced images. Although we enhance the images after Gaussian noise corruption and lowering illumination, the quality of the input images is still low in most cases. AudioCSRNet is able to generate high-quality density maps on low-quality images, however, when people in highquality images are recognizable, AudioCSRNet could perform slightly worse (see the examples shown in the fourth and last rows).