{'fraction_non_alphanumeric': 0.033577442335475514, 'fraction_numerical': 0.013002540726348827, 'mean_word_length': 4.83546511627907, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Object Detection using Deep Learning: A Review

Object detection is one of the most critical and challenging tasks in computer vision. It is the process of finding objects belonging to some predefined categories and determining their location in an image or video. This paper reviews deep learning-based object detection models. The paper discusses some benchmark datasets. The performance evaluation of different detectors on different datasets based on mean Average Precision (mAP) is reviewed. Object detection is used in different fields in different forms. Applications of object detection like pedestrian detection, autonomous driving, face detection, etc., are presented. Finally, the future scope is discussed to work on new techniques for object detection.

Object detection can be divided into three types: (i) generic object detection, (ii) salient object detection, and (iii) categoryspecific object detection [23]. Generic object detection detects objects of generic type like a person, an animal, a car, etc., in the image. There can be significant challenges to detect all objects in an image or video as same category objects can have different forms or orientations. For example, alligators and crocodiles belong to the animal category, and they have a similar appearance and can be difficult to detect accurately. Salient object detection is inspired by human visuals as it aims to detect the region which is more attentive from the surrounded area in the given image. Salient object detection (SOD) can be used as preprocessing step in many computer vision tasks, such as image retrieval, object segmentation, etc. [23]. The third type, category-specific object detection (COD), detects specific instances of predefined object categories in the image. For example, detection of Albert Einstein's face in the images. The focus of this paper is on generic object detection.
The organization of the paper is as follows. Section 2 presents a discussion on object detectors. To determine the model's performance, evaluation metrics such as precision, recall, mean Average Precision (mAP) are used and discussed in Section 3. Pascal VOC, MSCOCO, Open Images are benchmarks that are extensively used in generic object detection. The number of images and the number of categories used in benchmarks are discussed in Section 4. Finally, Section 5 concludes the discussion and presents future directions.

Detectors in Object Detection
Existing image object detection detectors are categorized into two parts: "two-stage detectors" and "one-stage detectors." The difference between these two detectors is the presence of a region proposal in the two-stage detector. Two-stage detectors are known to be more accurate for detecting objects, but on the other hand, one-stage detectors are fast but less accurate [2]. R-CNN, Fast RCNN, Mask RCNN are two-stage detectors, and onestage detector consists of YOLO, SSD, RetinaNet, and many more. The basic architecture of the one-stage detector and two-stage detector is explained in Figure 1 and

Two-stage detectors
Two-stage detectors divide the object detection task into two parts: (i) Region proposals (ii) class prediction. In the first phase, the detector tries to determine the regions which may contain the object. In the second phase, the object detector classifies the object present in the proposed region into different categories [8]. Some two-stage detectors in deep learning are R-CNN, SPP-Net [24], Fast RCNN, Faster RCNN, RFCN [25], etc.

R-CNN
Girshick et al. proposed the R-CNN [11] method to enhance the quality of candidate bounding boxes as region proposals [26]. R-CNN uses selective search to generate region proposals. It obtained 53.3% mean average precision, 30% more than the previous best results on PASCAL VOC 2012. Previous models detect objects based on traditional approaches, which are Haarlike features, HOG, etc. R-CNN-based object detection consists of three stages: (i) Proposal Generation, (ii) feature extraction, and (iii) classification [8]. Selective search generates about 2000 region proposals for each image and rejects regions identified as background regions. Then these region proposals are resized into fixed-sized regions and converted into 4096-dimensional feature vectors by the deep convolutional neural network. These extracted features are used to teach bounding box regressors to produce final proposals that contain the object. R-CNN adopts weights of network pre-trained on the ImageNet dataset. The last fully-connected layer was initialized again for the detection part.
To decrease false positives and enhance learning speed, R-CNN rejects easy negatives [8]. The threshold of 0.5 is defined for Intersection over Union (IOU) to set the examples as positive or negative. The region proposals having a threshold greater than 0.5 are considered positive, while below 0.5 are considered negative. Despite various advantages, R-CNN has few disadvantages also: • Heavy duplicate computation is performed because the proposal features are extracted separately.

SPP-Net
A fully connected layer works on fixed-size input. Because of this, the region proposals are cropped to a fixed size. However, the region of interest may be of any ratio, and the information may lose while cropping the image. CNN architecture named SPP-Net [24] is introduced to overcome this problem, which uses Spatial Pyramid Matching (SPM). Instead of cropping the image, the feature map is calculated for the whole image. The fixed-size feature vector is extracted from the feature map using the spatial Pyramid Pooling (SPP) layer. The feature map is divided into N x N grid, and the feature vector is extracted by performing pooling on each grid [8]. Then SVM classifier and the Bounding Box Regressor take the extracted feature vectors as input. There is no information loss in SPP-Net, and it has better efficiency than R-CNN because it can work on different image scales.

Fast R-CNN
SPP-net achieves better results than R-CNN; however, it still has some cons. SPP-Net uses the same pipeline as R-CNN to detect the objects. It requires a large amount of space. Fast R-CNN [12] uses the Region of Interest (RoI) Pooling layer to extract the region features. Feature maps are produced from the whole image, and then from each region proposal, a fixed-length feature vector is extracted with the RoI pooling layer.
Further, feature vectors are fed to two output layers: the classification layer and the other one is the regressor layer [8]. Unlike R-CNN, all three steps are optimized end to end without taking extra space to store the features in Fast R-CNN. Fast R-CNN has a better inference speed and can perform detection with greater accuracy than SPP-Net and R-CNN. It obtained 66.9% mean average precision (mAP), greater than R-CNN with 66.0% mAP on the PASCAL VOC 2007 dataset.

Faster R-CNN
Detectors till Fast R-CNN shows improvement in accuracy, but they all use traditional approaches for proposal generation such as selective search or slow edge boxes. To overcome this, Faster R-CNN was proposed after three months of Fast R-CNN proposal [2]. Faster R-CNN [13] generates region proposals using Region Proposal Network (RPN), which efficiently predicts region proposals with different scales and various aspect ratios. RPN takes the arbitrary size of an image, and object proposals are generated on each position of the feature map by sliding over the feature map using the N x N grid. The feature vector is then fed into two layers, the same as in previous detectors [8]. RPN uses anchor boxes to generate region proposals of different aspect ratios. Faster R-CNN achieves high precision and detection efficiency. It achieved 69.9% mAP on PASCAL VOC 2007 with a frame rate of 5 frames per second compared to Fast R-CNN with 66.9% mAP [2]. The different regions share feature extraction computation among them. However, the classification step does not share this computation in which each feature vector goes through the sequence of the fully connected layer. It increases the computation cost as one image can have hundreds of region proposals.

R-FCN
The problem of non-sharing of computation among classification steps is solved by proposing a Region-based Fully Convolutional Network. R-FCN [25] encode relative position information of different classes using Position sensitive Score Map, and spatial aware region features are extracted by Position-Sensitive RoI Pooling layer (PSROI pooling layer) [8]. It results in achieving better results than Faster R-CNN. Faster R-CNN has difficulty in detecting objects at different scales because it uses a single deep layer feature map. Another method is proposed to address this problem, especially for scale invariance, called Feature Pyramid Network (FPN) [27]. It makes the computation time and memory increase rapidly.

Mask R-CNN
Mask R-CNN [14] is proposed for object detection and instance segmentation by replacing Faster R-CNN RoI with RoIAlign [1]. RoIAlign is designed to preserve pixel-level spatial information [28] and extract a small feature map from each RoI. The quantization operation performed in traditional RoI pooling operation cause misalignment between extracted features and RoI. To overcome this problem, Mask R-CNN uses the RoIAlign layer, which is quantization-free and simple. This change brings a significant improvement in the model accuracy. Mask R-CNN used ResNeXt101-FPN as the backbone network and achieved the highest results for the COCO dataset for object detection and instance segmentation.

One stage detectors
Instead of using two stages like two-stage models, one stage model detects the object in one step. It does not have a separate stage for region proposals, but it considers every position as a potential object and then classifies it as a target object or background. Two-stage detectors are not very suitable for realtime detection because of the more time taken for detection. Onestage detectors take less amount of time, which is why they are used for real-time detection. So the detectors like YOLO, SDD are widely used nowadays. These methods adopt the idea of regression in which the input image is divided into cells, and each cell determines the class probability [9].

YOLO
Yolo [15], You Only Look Once, is the one-stage object detector that can perform object detection by applying a single neural network to the whole image [5]. YOLO was proposed to use a whole feature map to predict confidences for multiple classes. The image is divided into an S x S grid, and the object centered in the grid is detected in each grid, and the confidence score is calculated. How likely an object is present in the grid is formally known as the confidence score. The conditional probability p(classi|object) is calculated for each bounding box, which is the probability of an object belonging to a particular class among multiple classes. YOLO is faster than the previous two-step models and runs at 45 FPS in real-time. YOLO is less likely to detect false positives from the image. YOLO sometimes fails to detect small objects in the image because only one object can reside in one grid cell. It predicts the five values in the image: image x, y coordinates, width and height of bounding box, and objectness [4]. YOLO gives 63.4% average precision on the VOC2007 benchmark. Despite high speed than previous twostage models, it has various disadvantages (i) it is difficult to detect small objects because, at a time, only up to two objects can be detected in a given location. (ii) Prediction of objects uses only the last feature map that cannot detect objects of various aspect ratios [8].
YOLOv2 [29] is the improved version of YOLO with enhanced speed and precision. YOLO uses a fully connected layer to predict the coordinates of the predicted boxes, but YOLOv3 [30] replaces this fully connected layer with anchor boxes, increasing the recall by 7% [2]. It also comes with various advancements such as batch normalization, dimension cluster, and multiscale training [26].

SSD
Single Shot Multibox Detector (SSD) [16] came in 2015 to predict the bounding boxes and offsets using anchor boxes for default bounding boxes at different scales and different aspect ratios [23]. The performance improved with multiscale feature maps and default boxes. The SSD detector also detects small objects that were difficult to detect in YOLO. Authors also use hard negative mining and data augmentation to improve the precision of the model [2]. It achieved 74.3% mAP on VOC2007 at 59 FPS, more significant than the two-step model RCNN and YOLO [28].
The prediction module and deconvolution module are added to SSD, making a different model called Deconvolutional SSD (DSSD) [17]. Resolution of feature map increases with the deconvolution module. Both modules predict objects with different sizes, enhancing mAP with 2.2% on PASCAL VOC 2007 test dataset [2].

RetinaNet
Despite various advancements in one-stage detectors, they had less accuracy than two-stage detectors. With the introduction of focal loss in RetinaNet [18], the model's accuracy was improved while maintaining high detection speed [4]. Focal loss is the proposal of the loss function, which outweighs background examples and focuses on foreground examples [4]. Class imbalance and inequality between the types of training examples was the reason behind the lesser accuracy of one-stage models. It occurs because one-stage detectors focus on the whole image, resulting in more samples of the background region. RetinaNet works at around 60 FPS [4].

Evaluation Metrics
Different evaluation metrics are used in object detection tasks to evaluate models. One of the important metrics is Average Precision (AP). The basics which are used to calculate AP are: There can be numerous bounding boxes that need not be detected, so True Negative (TN) is not considered to evaluate performance [33]. Based on the predicted box (Oi), the prediction can be correct or incorrect, which is decided using Intersection over Union (IOU) by comparing it with the ground truth bounding box (Bi).
If this value is greater than the predefined threshold α set as 0.5, then the prediction is correct otherwise incorrect.

Precision and recall
Precision and recall can be calculated as Where precision is correct detections for all detections by model and recall is correct detections from all ground truth boxes. The tradeoff between precision and recall is generally present in the model. When the FP is low, its precision will be high, but as a result of this, the FN will be high, resulting in a low recall. A detector can be good if it has high precision and high recall [33].

Average Precision
The top 11 predictions of a model are used to calculate AP. For a known TP, precision and recall are calculated in each one of the 11 predictions. An IOU value greater than 0.5 is considered a correct prediction. Maximum precision is calculated for recall values ranges from 0 to 1. Averaging these precision values give Average precision for a detector [4].

Mean Average Precision
Mean Average Precision (mAP) is the average of all calculated average precisions over different classes. MAP is the most popular used metrics for performance evaluation, and it can be calculated as APx is the average precision of a particular class x, and C is the total number of classes [33].

Benchmarks
In the past years, several datasets and benchmarks are used to detect different kinds of objects. The most popular benchmarks are MS COCO, PASCAL VOC, ImageNet, and Open Images. PASCAL Visual Object Classes (VOC) [19] dataset has 20 categories: dog, person, cat, etc., and 11000 images with annotations. These images are further classified into different generic categories like animals, persons, household objects, and vehicles [1]. MS COCO [20] comes into competition in 2015 with 91 categories and 328000+ images divided into the training set, validation set, and testing set with 118287, 5000, and 40670 images, respectively [8]. ImageNet [21] has 20 categories with 450000 training images, 20000 validation images, and 40000 images for testing set [2]. Open Images [22] is another benchmark for detection, having 9.2M images with 600 object categories and 15440000 bounding boxes [5]. It is also used to detect the visual relationship between paired objects having some relation between them. The performance evaluation on different benchmarks with different models is given in Table 1 [1,28].

Applications
Object detection finds its applications in many disciplines like transportation, medical, security, military, etc. Some of the applications are presented below:

Pedestrian Detection
It is used to detect pedestrians as an object and further used in surveillance and for autonomous driving. Several datasets are built especially for pedestrian detection, like the Caltech pedestrian dataset [34], UCSD anomaly detection, and Penn-Fudan Database [35]. Pedestrian detection can be difficult because of the occlusion, small pedestrians, and some objects similar to pedestrians [26]. These problems can be tackled with the use of deep learning to a great extent.

Face Detection
Face detection is used to detect people's faces in an image, and it can further be used as preprocessing step for recognition and detection of facial expression. The first face detector was Viola and Jones [3], which used traditional features such as Haar-like features [26]. Face detection also has many challenges in detecting faces as there are many variations in people's faces with different poses and different expressions. Deep learning is currently being used for face detection as it is more accurate and less time-consuming than traditional features.

Autonomous Driving
Autonomous driving considers various aspects to drive without a human being. Person detection, traffic light detection, traffic sign recognition, building detection, and various other detections are used in autonomous driving. Sensor data is also used in driving to detect the location of the car. Because of no human involvement, it has various challenges and needs high accuracy for the model.

Text Detection
Text detection determines whether there is text present in the image or not. If it is present, then the location of the text is found, and then the recognition process takes place [26]. There are various challenges in text detection; for example, text may belong to any of the many languages used. Moreover, text can have different fonts, which introduces challenges in performing detection. Some text may have some missing characters, and some may have been written very densely, which is further challenging.

Discussion and Conclusion
With the increase in technology, previous models are evolving into better and powerful models to fulfil the growing needs. Over the past two decades, object detection achieves huge success in producing accurate and precise models. From two-stage detectors to one-stage detectors, both have their advantages and disadvantages. If time is of prime importance in the task, onestage detectors are a better option and can be used for real-time detection. The execution speed of these detectors is higher than two-stage detectors. However, if the requirement is high accuracy and computational cost is not an issue, a two-stage detector can be used because of its higher accuracy [28]. There is no superior model; based on the requirements, the detectors can be chosen to balance the tradeoff between speed and accuracy. Future work can be initiated on various aspects, such as detecting small objects, which is still an issue with current models. Object detection and activity recognition can be performed considering the correlation between video frames. Generative Adversarial Network is an emerging area and can be explored for object detection and activity recognition. This paper has presented the different models used for object detection, including two-stage and one-stage detectors and various performance evaluation metrics.