{'fraction_non_alphanumeric': 0.02109375, 'fraction_numerical': 0.0056640625, 'mean_word_length': 5.180446590223295, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Privacy Aware Person Detection in Surveillance Data

Crowd management relies on inspection of surveillance video either by operators or by object detection models. These models are large, making it difficult to deploy them on resource constrained edge hardware. Instead, the computations are often offloaded to a (third party) cloud platform. While crowd management may be a legitimate application, transferring video from the camera to remote infrastructure may open the door for extracting additional information that are infringements of privacy, like person tracking or face recognition. In this paper, we use adversarial training to obtain a lightweight obfuscator that transforms video frames to only retain the necessary information for person detection. Importantly, the obfuscated data can be processed by publicly available object detectors without retraining and without significant loss of accuracy.


Introduction
In the past years, we have seen a staggering amount of surveillance cameras deployed in our streets, shops and cities [1]. While these cameras are installed for legitimate reasons, there is an intrinsic risk that the video data will be used for unintended purposes [2]. Reducing this risk of function creep is especially important when the video analysis is offloaded from the camera to a third party platform using a Software as a Service (SaaS) model. Using SaaS is often preferred as object detection can be too computationally expensive for edge devices and third party software gives better chances at state of the art results. In this paper, we obfuscate surveillance video frames to hide identity revealing information while still allowing for accurate person detection. Crucially, our method of data obfuscation does not require adaptations or retraining of the downstream detector models and generalizes across cameras and object detectors.
We use adversarial training to obtain an obfuscator model to transform video frames in such a way that object detection can still be performed with high accuracy but no other tasks. Furthermore, we propose to do this obfuscation using a computationally efficient model that is suitable for execution on the camera. We then transmit the obfuscated data to a cloud backend to perform the computationally expensive object detection.
We focus on inference phase privacy, thereby assuming access to a pretrained object detection model. A powerful technique for this is Homomorphic Encryption (HE) which can perform calculations on encrypted data without ever decrypting it [3,4]. A disadvantage of HE is its large computational cost [5].
An alternative technique is called Secure Multi-party Computation (SMC). It ensures privacy by splitting computations over several computing parties [6]. SMC is mainly fit for simple functions, usage for complex functions such as neural networks is possible but similarly to HE often results in heavy computation costs [5].
A promising category in inference phase privacy are data transformation techniques. Their main goal is to limit the transmitted data to the bare minimum required for the task. Research includes Leroux et al.'s [7] adversarial obfuscator which forms the basis for this work and Mireshghallah's Shredder [8]. Shredder splits a network in two parts in which one part runs on the cloud and the other on a users device. Before sending data from the users device to the cloud, noise is added to ensure privacy. A disadvantage of Shredder is that it requires modification of the downstream model.
Other work focuses specifically on privacy aware people detection through face anonymization [9,10]. This however, requires an often computationally expensive face detection method to be executed on the edge device and could possibly still allow a persons identity to be retrieved due to other attributes (clothing, sex, haircut) remaining visible [11].
In this paper, we build upon our previous work [7]. Our current contribution scales the technique up to real-world surveillance data and provides experimental benchmarking with other obfuscation methods. Furthermore, we assess its generalization performance across multiple cameras and across object detector models.

Privacy preserving obfuscation
Our architecture is shown in Figure 1 and consists of an obfuscator and deobfuscator model that are trained in opposition to each other, similar to Generative Adversarial Networks (GANs) [12]. The obfuscator learns to transform frames in such a way that the deobfuscator cannot reconstruct them while ensuring a pretrained object detection model can still make accurate predictions. The deobfuscator's only goal is to reconstruct obfuscated frames to their original form. At test time, the deobfuscator is discarded. Details on the training procedure can be found in the appendix. The obfuscator and deobfuscator are both autoencoders based on the MobileNet [13] architecture. This architecture is designed for lightweight and fast models to be executed on resource constrained devices.
First and foremost, the goal of our technique is to obfuscate video frames in such a way that their contents are no longer usable for person identification. The original, obfuscated and reconstructed version of a sample frame can be seen in Figure 1. In the obfuscated image, the persons have been transformed into silhouettes which are still correctly classified by the object detector but do not allow for identification. The results of the deobfuscator indicate that accurate reconstruction of the frames is impossible. However, further work is needed to prove the robustness of these claims.

Experiments and results
All experiments have been implemented using the PyTorch [14] library. Unless stated otherwise we used the pretrained Faster R-CNN [15] with ResNet50 backbone as our object detection model. We used street surveillance videos from the Avenues [16] and ShanghaiTech Campus (STC) [17] datasets from which individual frames have been extracted and scaled to (320 × 200).

Detection accuracy
As the datasets were originally intended for anomaly detection, we consider the detections of a pretrained Faster R-CNN as ground truths. We report the Average Precision (AP) of person detections by Faster R-CNN on the obfuscated frames in Table 1. We show only the person AP as our target task is people counting. To investigate generalization, we conducted experiments with obfuscators trained on the STC and the Avenues dataset. Further, to investigate the robustness across multiple cameras, we have split the STC dataset into two sets containing footage of different cameras (camera 1-8 and 9-13). The results indicate that models can generalize over different cameras, perspectives and illumination levels. However, the accuracy drops significantly when testing on a different dataset than it was trained on. This could be attributed to the large amounts of passengers in the Avenues dataset in comparison to the STC dataset. A question one could pose is whether the obfuscator is overfitted to the object detection model it was trained with. Ideally, the obfuscator retains general features that all object detection models use. Table 2 shows the accuracy reached when the obfuscated frames are classified by another detector than the one used at training time. The accuracy remains fairly high when testing on different models. Even when testing on YOLO [18], a model with an entirely different architecture than the region based models, accuracy remains high. This generalized obfuscation means that users of third party object detection models can obfuscate their data without having to worry about the precise model used by the SaaS provider.

Obfuscation
We also compare our approach with traditional techniques for obfuscation such as blurring, adding noise or quantizing the color information. We found that these techniques had a significantly worse accuracy-obfuscation trade-off. This is illustrated in Table 3 on frames from the Avenues dataset.
To ensure a fair comparison, the parameter values of the obfuscation techniques were set to achieve similar average precision of person detection than our model. We quantitatively measure the quality of the obfuscation by calculating various similarity metrics between the original and obfuscated image: the Structural Similarity Index Measure (SSIM) [19], Mean Squared Error (MSE), Peak Signal to Noise Ratio (PSNR) and Normalized Mutual Information (NMI) [20]. Except for the MSE, a lower score indicates a stronger obfuscation.   Figure 2: Accuracy in function of MACs, the obfuscator can be significantly reduced without large accuracy loss.

Computational efficiency
Leaving aside the operational and economic benefits of using a object detector offered as a cloud service, from the perspective of energy efficiency an obvious alternative is to run an object detector on the camera and only transfer the counts to the cloud. The footprint of a neural network is typically measured by means of the number of multiply-accumulate operations and the number of parameters to be stored in memory. In Table 4 we compare the size and computational cost of the obfuscator to those of two common object detection models. The obfuscator outperforms the evaluated object detection models in terms of multiply-accumulate operations (MACs) and parameters. By reducing the amount of convolutional filters in each layer, we can reduce the MACs of the obfuscator. The corresponding degradation in accuracy is shown in Figure 2. Even after reducing the MACs with a factor 20, the detection accuracy is still above 70%.

Conclusion
We have shown that our adversarial obfuscator structure can be successfully modified for the person detection problem. Our obfuscation of video frames results in a minimum degradation of person detection accuracy and generalizes well across camera viewpoints and object detector models. Furthermore, the obfuscation is suitable for deployment on the edge. Future work will focus on more formally proving the irreversibility of the obfuscation and on the robustness against function creep by an information-theoretic analysis of the mutual information between the obfuscated frame and the features required for other classification tasks.