{'fraction_non_alphanumeric': 0.024029677205027408, 'fraction_numerical': 0.001827141354299319, 'mean_word_length': 4.873821138211382, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Edge detection based on Improved Non-Maximum Suppression Method

Semantic Segmentation and edge detection are important research fields for scene understanding in computer vision. A hierarchical framework called Contextual Hierarchical Model (CHM) was proposed for semantic image segmentation and edge detection. It learned contextual information using a Logistic Disjunctive Normal Networks (LDNN) classifier. The class average accuracy of CHM was improved by defining a global constraint using Conditional Random Field (CRF), Hierarchical CRF (HCRF), Higher order HCRF (HHCRF). The LDNN was improved by using proximal gradient method which minimizes the quadratic error and it had high convergence rate than gradient descent method. The weight and bias terms of LDNN was optimized by using Grey Wolf Optimization algorithm (GWO) which improves the classification accuracy and it also reduces time complexity of LDNN. During the edge detection using CHM, a multi-scale strategy was adopted to compute edge maps. A Non-Maximum Suppression (NMS) was used to obtain the thinned edges in images. However, it is a post-processing step which consumes additional time in edge detection process. So, in this paper, the edge detection is interpreted as a classification problem where the thinned edges in images are obtained without any post-processing step. It reduces the time consumption. Two key ingredients such as loss and joint processing are included in NMS to improve the detection of thinned edges. The loss is used to penalize the double edge detection and the joint processing is used to reduce the loss of edge detection by including a pair features as additional feature for edge detection. The pair features, Haar, Histogram of Gradient (HOG) and SIFT features are given as input to LDNN to detect the edges that improve the efficiency of CHM based edge detection.


I. INTRODUCTION
Image segmentation [1] is the process of partitioning an image into number of regions or set of pixels. The outcome of the image segmentation process is a set of contours or regions that collectively covers the whole image. Based on some characteristics or computed property, such as intensity, color or texture the pixels are grouped in a region. Neighboring regions are significantly dissimilar with respect to same characteristics. Semantic image segmentation process [2] not only realizes what objects are in an image but it also used to locate objects in an image. It divides an image into a finite number of non-overlapping and meaningful regions. Edge detection is one of the important processes in semantic image segmentation.
Edge detection [3] is the process of finding and locating sharp discontinuities in an image. Often, the edges of object planes in an image provide the oriented localized changes in the intensity of that image. Edge detection techniques convert an original image into edge image assisting from the changes of grey tones in the images. The outcome of this transformation returns an edge image without changing any physical qualities of the original image. Based on the detected edges, the objects in images are segmented.
A contextual framework called contextual hierarchical model (CHM) [4] was proposed for semantic image segmentation. The contextual information of an input image was learned in a hierarchical framework. It consists of multiple hierarchy levels where a Logistic Disjunctive Normal Networks (LDNN) was trained at each level of hierarchy based on down sampling input images and outputs of previous levels. Then multi-resolution contextual information from each level was combined into a classifier to segment the input image at original resolution. Even though CHM is able to model contextual information within a scene, it can be prone to errors due to the absence of global constraints. Due to the absence of global constraint, the CHM has poor performance in terms of class average accuracy.
So, the global constraint for CHM was defined through different models such as Conditional Random Field (CRF), Hierarchical CRF (HCRF) and HCRF with higher-order features (HHCRF) [5] where energy functions were defined on a discrete random field. The CRF, HCRF, and HHCRF were processed in the bottom-up step of CHM and its result was utilized to train LDNN. It segmented the images semantically within the top down steps of CHM. A gradient descent was utilized in LDNN to minimize the quadratic error. It has slow convergence rate which can affect the efficiency of LDNN.
So, CHM-HHCRF-Improved LDNN (CHM-HHCRF-ILDNN) [6] was proposed where a proximal gradient method is used to minimize the quadratic error which has a faster convergence rate. Moreover, CHM-HHCRF-Improved Optimized LDNN (CHM-HHCRF-IOLDNN) was proposed to optimize the weight and bias term of LDNN using Grey Wolf Optimization algorithm (GWO) [7] which improves the classification accuracy. The thinned edges in the images were detected by CHM-HHCRF-IOLDNN using standard Non-Maximum Suppression (NMS). NMS is processed as a post-processing and it takes a longer time to detect the edges.
So, in this paper, a CHM is designed without any post processing for edge detection. The edge detection based on NMS is improved by including two key 0. Such as, loss and joint processing to build an NMS. A loss that penalizes double edge detection and the joint processing of edge detection reduces the loss of edge detection. In the joint processing, a pair of features in images are included and it is given as additional feature along with the features such as Haar, Histogram of Gradients (HOG) and SIFT to CHM-HHCRF-IOLDNN for edge detection. Hence, NMS without post-processing reduces the time consumption and two key ingredients improve the accuracy of CHM-HHCRF-IOLDNN based edge detection.

II. LITERATURE SURVEY
Based on Neutroscopic Set (NS) structure, an edge detection method [8] was proposed. This method was started with split an image into T, I and F subsets. Then, these subsets were undergoing the edge detection process. The edges in the images were related to the indetermination situations. So, NS was utilized to analysis the edges of images. T and F subsets were identified by using entropy those subsets were considered as significant elements in edge detection with NS. However, the entropy used in this method has an impact on the edge detection results.
Using a modified Moore-Neighbor algorithm, an edge detection method [9] was proposed. It is also called as 8-Neighborhood. For boundary detection and feature extraction, the modified Moore-Neighbor algorithm was used as boundary tracing. In this algorithm a set of 8 pixels shared its vertex or edge with a pixel. In order to detect the edges of objects in the images, a range of filter was integrated with the modified Moore-Neighbor algorithm. However, this method still has poor Baddeley's Delta Metric at some point of variance.
For edge detection, a novel Convolutional Neural Network (CNN) based pipeline [10] was presented. It integrated multi-level information in a feature map-based manner. The multi-level information was extracted from different intermediate layers creating Hybrid Convolutional Features (HCF). These features were fed into the edge detector which detected edges without post-processing. Although this method achieves 22fps speed, it is not a real edge detector.
Based on feature re-extraction (FRE) of a deep convolutional neural network, an edge detector [11] was proposed for edge detection of objects in images. This edge detector was comprised of backbone, side-output and feature fusion modules. The backbone module extracted preliminary features from an image. The side-output module applied residual learning to make network architecture more robust by mapping features from various stages of the backbone network with edge-pixel space. The edge map was generated in the feature fusion module. However, the computational complexity of this edge detector is high.
Based on Computational Ghost Imaging (CGI), an optical edge detection method [12] was proposed for edge detection. It recognized the features of an object in the image. The CGI with standard illumination was generated by using an interference system. Furthermore, structured intensity patterns were designed to directly map the detected data in CGI with the edge of an object. The optical edge detection method extracted the boundaries for grayscale and binary objects in any direction at one time. However, this method extracted boundaries sharp and clearly visible only up to deviation 5mm.
For semantic image segmentation, a hybrid Bayesian Network (BN) model and Hierarchical Conditional Random Field model (HCRF) [13] were proposed. HRCF generated initial semantic sub-scene prediction by capturing non-casual relationship whereas BN modeled contextual interactions for each semantic sub-scene. From the training data, the conditional probabilities of contextual interactions were. Then, the structure of contextual dependencies was encoded in the initial predictions from learned BN structure to create final refined predictions. However, this method has low accuracy. For semantic image segmentation of remote sensing harbor images, an edge aware deep convolutional neural network [14] was proposed. The segmentation and edge detection networks were trained simultaneously by designing a multitask model. The edge network was learned from the extracted hierarchical semantic features. The entire model was further refined by adding the outputs of edge pipeline with an edge-aware regularization. However, a user-defined parameter was used to adjust the weight of regularization term that greatly influences the segmentation results.

III. METHODOLOGY
In this section, edge detection using standard Non-Maximum Suppression (NMS) and NMS without post-processing are described in detail. The post-processing consumes more time and hence the edge detection process takes a longer time to detect the thinned edges. So, a CHM is designed with NMS without post processing and two key ingredients such as loss and joint processing are introduced in NMS to improve the efficiency of edge detection.
A. Standard NMS method for edge detection Non-Maximum Suppression (NMS) works by finding the pixel with the maximum value in an edge. NMS is carried out to preserve all local maxima in the gradient image and deleting everything else which results in thin edges. NMS consists of:  Consider a point , where and are pixel integers and be the intensity of pixel .  Calculate the gradient of image intensity and its magnitude in .  Estimate the magnitude of the gradient along the direction of the gradient in some neighborhood around .  If is not a local maximum of the magnitude of the gradient along the direction of the gradient then it is not an edge point.

B. NMS without Post-processing
The standard NMS gets input as window, window associated edge class labels and window network response score. Then it removes those windows which are not locally the highest-scored, to yield a final set of edge detections. It is processed as a post-processing step in CHM to obtain the thinned edges. It consumes additional time to detect the edges. In order to reduce the time consumption, learn the NMS into the classifier. For every possible detection in an image, it computes the probabilities of edge classes. It gives rise to score detectors which develop a search space of detection. Then class probabilities were computed for each edge detection. Accordingly, two strongly overlapping edges in the image will both result in high score since they look at almost identical image content. Based on the edge coverage capacity of detection windows, each contents in the image triggers various edge detections of varying confidence.
NMS creates high confidence edge detections in which it is considered that the highly overlapping edge detections belong to the same image pixel and combine them into one edge detection. The IOLDNN classifier differentiates the image content that contains an edge and the image content that does not contain an edge. In the training data of IOLDNN for edge detection, the positive and negative samples are defined by some measure of overlap between edges and windows. Since similar window will generate similar confidence, a small perturbation of edge locations is considered positive samples. However, the trained IOLDNN classifier does not provides one high scoring edge detection. It encourages multiple high scoring edge detections.
In order to generate exactly one edge detection, two key ingredients are considered. One of the ingredients is a loss that penalizes double detections to teach the detector precisely. Another ingredient is the joint processing of neighboring detections so the detector has the necessary information to define whether an edge is detected multiple times.
IOLDNN based edge detector is supposed to return exactly one high scoring edge detection. The loss for such a detector must restrain multiple edge detections of the same image pixel, irrespective of how close these detections are. A matching strategy is used to ensure the results of NMS based multiple edge detections. The IOLDNN detector is judged by the evaluation criterion of a benchmark, which in turn describes a matching strategy to decide which edge detections are correct or wrong. This strategy is used during the training stage of the edge detection. Based on the edge detection confidence, benchmarks sort detection in descending order. Then, in the sorted order match the edge detection to image pixels. In this matching strategy, already matched image pixels cannot be matched again and the remaining edge detections are counted as false positives that decrease the precision of IOLDNN based edge detection. The results of this matching strategy are used as labels for the IOLDNN to detect the thinned edges.
All detections that are used for training of a classifier have a label as they are given as input to the IOLDNN network. In such situation, this network has access to edge detections and image pixel annotations. The matching strategy creates labels that depend on the prediction of the network. Consider is a detection, indicate whether is successfully matched or not to an image pixel and denotes the scoring function that jointly scores all detections on an image , where denotes the number of edge detections. This is trained with the weighted logistic loss In the above equation, matching strategy is used to couple the loss per edge detection with the other edge detections. The matching produces and is used to counter act the extreme class imbalance of the edge detection task. The IOLDNN detects the thinned edges by minimizing the weighted logistic loss.
Each IOLDNN consists of edge detections on an image each edge detections are represented by a dimensional feature vector. A detection context layer is used for every detection , generates all pairs of detections for which sufficiently overlaps with . The representation of a pair of detections consists of the concatenation of both detection representations and -dimensional detection pair features which yields an dimensional feature. The pair features consist of several properties of a detection pair are the intersection over union, the normalized distance in and -direction and the normalized distance, scale difference of width and height and difference of aspect ratio are used as pair features in pairwise context. The features of all pairs of detections are arranged to process each pair of edge detections independently. If edge detection has neighboring edge detection that yields a size , where since the pair is also included. The max pooling is used to reduce the variable sized neighborhood into a fixed size representation. The pair features of pairwise context are also given as additional input to the IOLDNN which improves the efficiency of the edge detection process.  Table I. As shown in Table I and Fig. 1, the proposed CHM without post-processing (CHM-HHCRF-INMS-IOLDNN) model achieves the state-of-the-art performance. In CHM-HHCRF-IOLDNN, the thinned edges are not detected. In the CHM-HHCRF-NMS-IOLDNN model, NMS is done as the post processing process to detect the thinned edges. In order to have true end-to-end learned detectors, interested in developing detectors without any post-processing. The detectors with post-processing has impact on the performance in terms of average precision which is clearly shown in Table I Table II.     Table III  In addition to this, NMS with loss and joint processing for penalizing the double edge detection and to check whether the edges are detected multiple times enhance the pixel accuracy of edge detection process. When the number of levels in CHM is 1, the pixel accuracy of CHM-HHCRF-INMS-IOLDNN is 2.17% more than pixel accuracy of CHM-HHCRF-IOLDNN and 1.73% more than pixel accuracy of CHM-HHCRF-NMS-IOLDNN. From this analysis, it is concluded that the proposed CHM-HHCRF-INMS-IOLDNN has high pixel accuracy than the CHM-HHCRF-IOLDNN and CHM-HHCRF-NMS-IOLDNN.

V. CONCLUSION
In this paper, the NMS based edge detection in CHM is improved by including loss and joint processing in the edge detection process. Initially, the edge detection is deduced as a classification problem. The NSM based edge detection method was providing multiple detections for a single object. In order to provide a single edge detection of an object with a high score, a weighted logistic loss is included in NMS. In the joint processing, concatenates the detection representation and pair features which yield new dimensional features. It is given as additional input to LDNN along with Haar, Histogram of Gradient (HOG) and SIFT features which enhance the performance of NMS based edge detection in CHM. The experiments are conducted in Berkley Segmentation Dataset that shows that the proposed CHM-HHCRF-INMS-IOLDNN based edge detection method has better performance in terms of average precision, ODS and pixel-wise accuracy.