{'fraction_non_alphanumeric': 0.025920498259754534, 'fraction_numerical': 0.010899432130426817, 'mean_word_length': 4.921366594360086, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Energy Efficient Automatic Streetlight Controlling System using Semantic Segmentation

This study aims to develop a novel streetlight management system powered by computer vision technology mounted with the close circuit television (CCTV) camera that allows the light emitting diode (LED) streetlight to automatically light up with proper brightness by recognizing the presence of pedestrians or vehicles and reversely dimming the streetlight in their absence by semantic image segmentation from video.


Introduction
Street lighting system plays a vital role in the society particularly, in school campuses. It enables vehicular movement at night; reduced the associated risks of nighttime accidents; assist protection of property; and makes students feel secure [1]. Despite of those benefits, streetlights also contribute to big energy consumption and give a big impact in the countries energy budget. Many studies came up with designing a street lighting system that can minimize its power consumption in recent days as the worldwide energy crisis has emerged enormously than ever. Aziera et al. [2] developed a prototype using Light Dependent Resistor (LDR), Infrared sensor (IR), battery and LED that has the facility to control the brightness to reduce the power consumption. Likewise, the approach from Mary et al. [3] consists of similar sorts of sensors and a controller which make it as an intelligent street lighting system. Islam et al. [4] also used IR sensors to detect vehicle movement and pedestrians with an additional facility of detecting road accidents and notify the concerned authority by sending the location and car number. Furthermore, Arun et al. [5] describe a method through their proposed work with PIR and LDR sensors to sense the incoming traffic and hence turning ON a cluster of lights surrounding the traffic as well as the day time when there is no need of light correspondingly to conserve power. In the same way, other approaches [6][7][8][9][10][11] likely to be made using similar kinds of physical sensors, detectors and microcontrollers integrate with Internet of things (IOT) mostly. Despite of the existing designs, there is still a demand on streetlights in the market today for formulating an energy efficient system with less complication in physical design so that it requires minimal maintenance.
Semantic segmentation or image segmentation is the task of clustering parts of an image together which belong to the same object class using different convolutional neural networks (CNN). It has become a prominent technique in successfully solving many computer vision centric problems. This method has been used in localization [12][13][14], scene understanding [15][16][17], robotic navigation [18], detecting fire in surveillance system [19][20][21][22], roadside occupation surveillance system [23], autonomous vehicle driving [24][25][26] and many more. Being a least developed country, in Bangladesh the streetlight controlling and operating are mainly done through manual actions. Because of that most frequently it can be seen that the streetlights are being turned "on" before the evening time and not turned "on" when it is night time. The lack of supervision of the streetlight management is evident as the authority is not much concerned about the huge amount of wastage of energy in the long term for this negligence. Fig.1 refers our investigation photograph. The place of taking the photo is in Chattogram which is the second largest city in the country. At the time of 5 p.m. in the afternoon, the streetlight is turned "on" before it is being dark. Hence, due to the improper nature of streetlights management, the energy consumption is very high that results in an unnecessary luxury for the county. People in some parts of the country are deprived from electricity whereas vast amount of electricity has been wasted due to this poor street lighting system.
We wanted to assess the application of semantic segmentation to solve this practical problem in this regard. The next parts of the paper are organized as follows. Section 2 elaborates the proposed methodology of this article follows. Results and discussions are displayed in section 3. Finally, in Section 4, conclusion is presented.

The Proposed Framework
Motivated by the recent improvements in embedded processing capabilities and potential of deep features, we investigated numerous CNNs to improve the automatic streetlight controlling accuracy and minimize the false rate using semantic segmentation technique. An overview of our framework for automatic streetlight controlling in CCTV networks is given in Fig.2.
In our framework, from the real-time video feed of the CCTV camera the trained CNN model detects pedestrians and vehicles. If the model can recognize the presence of any kinds of objects of that sort, it ensures the streetlight is level up with 100% brightness. In reverse case, it dims the light to 50% or less. Again, the model measures the brightness of the day by pixel values of images getting from real-time video feed. By the given internal logic if it finds the brightness is below level, it turns off the streetlight and if it is above the level, it lights up the streetlight accordingly.

Dataset
We used two datasets to train two separate models for our framework. For the detection task of pedestrians and vehicles, we used Cambridge-driving Labeled Video Database (CamVid) 1 which is the first collection of videos with object class semantic labels, complete with metadata (Fig.3). The database provides ground truth labels that associate each pixel with one of 32 semantic classes including animal, bicyclist, car, road, pedestrian, sidewalk etc. There are over 700 images that were specified manually for the per-pixel semantic segmentation task. For the day and night detection task, we used a Kaggle dataset 2 named "day time and night time road images" that comprises with 14,607 day light images and 16,960 night time images.

Detecting Pedestrians and Vehicles
We utilized U-net architecture (Fig.4) for the task of detecting the pedestrians and vehicles which is an end-to-end network built upon the fully convolutional network [27], composed from encoder and decoder parts with skip connections. In order to get a precise segmentation and to localize high resolution features, these connections concatenates the input of each encoding stage input with its facing decoding stage input. We used ResNet-34 as backbone which plays the role of encoder in the network. The encoder part extracts features via several convolutions and ReLU activations to get the compressed features, then decompresses the features through the decoder part which contains deconvolutions and ReLU activations [27].

Detecting Day and Night
To detect and classify between day and night times based on the images from CCTV, we utilized the average pixel value of the image. If the average pixel value exceeds the value range of 100, the model detects that as day time and if reverse, it identifies the condition as night time. From Fig.5, we see some images that is a demonstration on how the applied image mask is applied to calculate the pixel values of the image and detect the time as day or night time on accordance.

Training Setup
As mentioned earlier, we utilized ResNet-34 as the backbone of our model. This is because ResNet-34 has demonstrated noteworthy performance as backbone in similar segmentation problems earlier [28,29]. The batch size was limited to 8 and epochs to 20. The learning rate has been set at 0.0001. We restricted the classes to be detecting by our model to only two classes, namely "car" and "pedestrian" as our interest is to detect them for the model we propose in this work. Softmax classifier has been used to classify between the object classes. We used the python library called "albumentation" to augment the data used for training as augmentation a powerful technique to increase the amount of your data and prevent model overfitting. We used different augmentation techniques such as, Gaussian noise, random crop, horizontal flip and so on as the training data was comparatively low and to train the model better..

Results and Discussion
We have run the experiment of training on the proposed model and demonstrate the results with evaluation matrices .

Intersection over Union (IoU) or the Jaccard Index
This is one of the most commonly used metrics in semantic segmentation. It is defined as the area of intersection between the predicted segmentation map and the ground truth, divided by the area of union between the predicted segmentation map and the ground truth: Where, A and B denote the ground truth and the predicted segmentation maps, respectively. It ranges between 0 and 1.

Mean-IoU
A popular metric which is defined as the average Intersection over Union (IoU) over all classes is known as Mean-IoU. In other words, the average area of intersection between the predicted segmentation map and the ground truth, divided by the average area of union between the predicted segmentation map and the ground truth: It is widely used in reporting the performance of modern segmentation algorithms. From Fig.3.a we can observe the mean-IoU score of our proposed model over the whole training period was recorded as 0.65, that is pretty satisfactory according to the study [30].

Loss
Categorical cross-entropy is used as the loss function of our model as shown in Eq. (3), which employs Softmax activations in the output layer. From Fig.7 we observe the loss of our model over the twenty epochs. The loss has been decreased over the entire training period. The final loss is 0.33.

F1-score
Another popular metric is called the F1 score, which is defined as the harmonic mean of precision and recall: The mean F1-score achieved by our model is 0.72.
In Fig.8, the results of the original masks and predicted masks of the proposed model are shown on three random test photos from the CamVid dataset.

Conclusions
This study demonstrated a revolutionary deep CNN based on the U-net that can dim streetlights as well as recognize vehicles and pedestrians using semantic segmentation of CCTV images. In compare to the conventional or the existing intelligent solutions, our proposed model is more affordable to set up, demands less energy, and doesn't require any physical sensors. Despite the fact that this study proposes an advanced and efficient streetlight management mechanism with improved precision even when using inexpensive components, further research in this area is still needed to support the validity of its realworld applications. Furthermore, the existing framework for controlling streetlights may be intelligently tailored for both controlling streetlights and conducting street surveillances. In real-world situations, this will make it possible for video surveillance systems to manage more complicated scenarios.