{'fraction_non_alphanumeric': 0.03540863820629598, 'fraction_numerical': 0.012687052700065062, 'mean_word_length': 4.746764452113891, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Confluence: A Robust Non-IoU Alternative to Non-Maxima Suppression in Object Detection

Confluence is a novel non-Intersection over Union (IoU) alternative to Non-Maxima Suppression (NMS) in bounding box post-processing in object detection. It overcomes the inherent limitations of IoU-based NMS variants to provide a more stable, consistent predictor of bounding box clustering by using a normalized Manhattan Distance inspired proximity metric to represent bounding box clustering. Unlike Greedy and Soft NMS, it does not rely solely on classification confidence scores to select optimal bounding boxes, instead selecting the box which is closest to every other box within a given cluster and removing highly confluent neighboring boxes. Confluence is experimentally validated on the MS COCO and CrowdHuman benchmarks, improving Average Precision by up to 2.3-3.8% and Average Recall by up to 5.3-7.2% when compared against de-facto standard and state of the art NMS variants. Quantitative results are supported by extensive qualitative analysis and threshold sensitivity analysis experiments support the conclusion that Confluence is more robust than NMS variants. Confluence represents a paradigm shift in bounding box processing, with potential to replace IoU in bounding box regression processes.


INTRODUCTION
BJECT detection algorithms that rely on Deep Convolutional Neural Networks (DCNNs) learn data-derived features that allow objects of interest to be localized in images [1]. State-of-the-art DCNNs return dense clusters of bounding boxes of varying sizes and locations, which congregate in regions likely to contain an object [2], [3], [4], [5], as illustrated in Fig. 1 and Fig. 2. To obtain the final detection output, only one optimal bounding box per object must be retained, and all excess bounding boxes must be removed. In cases where objects occlude each other, overlapping bounding boxes should be retained so that each object is represented by one bounding box.
This task is usually performed by variants of the Non-Maxima Suppression (NMS) algorithm [6], [7]. NMS uses the classification confidence score attributed to each bounding box by the DCNN to sort the bounding boxes in descending order. The highest scoring box is selected as the 'optimal' box for the first object. NMS then employs a user-defined Intersection over Union (IoU) threshold to suppress or decay the confidence score of all bounding boxes whose overlap with the 'optimal' box exceeds the threshold. This results in the removal of boxes that overlap heavily with the 'optimal' box. If there is more than one object in the image, NMS then selects the next highest scoring bounding box in the remaining set as the best box to represent the next object and repeats the suppression procedure. This process is repeated until the final detection set is attained -optimally, one bounding box per object.
NMS usually achieves acceptable performance in images where objects do not overlap each other much O Fig. 1. Confluence accurately selects an optimal bounding box for each person, with no retention of false positives. NMS selects a poorly localized maxima (confidence score: 96.4%), resulting in two optimal boxes surrounding the woman and boy (85.9% and 88.2% respectively) being suppressed by the IoU threshold. Due to high IoU, the bounding box surrounding the girl is also suppressed by NMS, reducing recall.
[8], [9]. However, when objects are occluded or overlap heavily such as in high density or crowded settings, reliance on IoU forces a trade-off between recall and precision [8], [9], [10], [11], [12]. This is because NMS is a heuristic algorithm that simply assumes that a high overlap between bounding boxes results in a high probability of one of the boxes being a duplicate [13]. Thus, to retain highly overlapping true positives, a higher IoU threshold must be used, leading to greater retention of false positives [14], [15], [16], [17], [18].
Another widely noted shortcoming of NMS is its poor localization accuracy occasioned by its sole reliance on the classification confidence score to select optimal bounding boxes [19], [20], [21], [22], [23], [24]. This score is a class label probability and does not effectively represent localization accuracy [24], exhibiting low correlation between localization accuracy and classification score [22]. Its usage by NMS means that NMS is susceptible to returning suboptimal bounding boxes, whilst suppressing better candidate boxes [21] as illustrated by Fig 1. It should therefore not be used as the primary metric by which the optimal boxes are selected.
Thus, this study presents Confluence, a novel non-IoU alternative to NMS algorithms. The key contributions are threefold. Firstly, we propose a proximity metric as an alternative to IoU in the suppression of false positives. Rather than using the IoU between bounding boxes to determine whether they represent the same object, we propose the confidence-weighted normalized pairwise Manhattan Distance [55] between corresponding bounding box coordinates to measure their coherence, and hence more accurately determine whether they point to the same object. Secondly, we propose Confluence NMS (NMS-C), a non-IoU variant of NMS, which retains bounding boxes using the classification confidence score, but suppresses false positives using the Confluence metric. Thirdly, we present Confluence, an algorithm which uses the proximity metric in both the selection of the optimal boxes, and the suppression of false positives. The effectiveness of Confluence and NMS-C is empirically validated against both G-NMS and S-NMS on the MS COCO [56] and CrowdHuman [57]

RELATED WORKS
Although NMS has been an algorithm of significant importance in computer vision for over 50 years, its shortcomings are as widely recognized as its essential role in the object detection pipeline [10], [58], [39], [40], [35], [11]. This has given rise to many adaptations of NMS, and various alternatives.
Many alternatives and adaptations aim to limit or eliminate the bottleneck caused by NMS in the object detection pipeline [59] by reducing computational expense and improving efficiency [60], [28], [61], [62], [63]. However, improvements in accuracy are achieved at the expense if performance, as these methods do not address localisation accuracy or recall [64].
Alternatively, some methods aim to improve proposal refinement during training to provide NMS with better input, thus improving performance [11], [24]. Other methods aim to circumvent NMS or eliminate the need for it altogether [65], [66], [67], [68], [69], [70], [9] however, they involve significant changes to neural network architecture, and achieve inferior or competitive performance [66]. This means they cannot be adopted in state-of-the-art object detectors [3], [5] that rely on NMS.
Some methods are specific to video [31], [71], body-part [72] or 3D [25], [73], [74] applications and are not applicable to or evaluated on standard object detection tasks. Other methods achieve gains in performance when compared to G-NMS and S-NMS but rely on additional data such as object depth information [17], or a combination of pixelbased and amodal bounding boxes [12], and corresponding network architecture changes, which are often not available or able to be implemented in standard object detection pipelines.
Most variants of NMS involve minor conceptual differences, resulting in similar performance. Matrix NMS was proposed by [64], which implements S-NMS in parallel for instance segmentation. Significant gains in speed were achieved, and it did outperform G-NMS but it did not outperform S-NMS and still relies on IoU and classification confidence scores. Similarly, [10] proposes GNet, a neural network that uses message passing between neighboring bounding boxes, whereby changes in bounding box representations are learned based on the 'negotiations' between bounding boxes to decide which bounding box will represent which object. Although this approach only uses bounding boxes and scores as input, it is a highly complex network which requires significant amounts of training data. In contrast, our approach does not require any training, and can be easily incorporated into systems that currently use NMS, without the need for architectural changes.
IoU-NMS [22] aims to improve bounding box localization by embracing IoU as the "natural criterion for localization accuracy". Initially, [22] proposes IoU-Net to learn the IoU relationship between ground truth box and the candidate boxes, returning a localisation confidence score per box. IoU-NMS then uses the localization confidence instead of the classification confidence score in the NMS procedure. Although [22] achieves higher Average Precision (AP) at higher IoU thresholds on several networks, it was not evaluated on standard benchmarks. Further, combining S-NMS with box-voting [75] returns similar results to [22] with reduced computational expense [21]. Notably, integration of IoU-Net in standard object detection pipelines is not easily achieved and requires retraining of object detectors, limiting its adoption.
Alternatively, a bounding box regression loss and a modified S-NMS dubbed Softer-NMS is proposed by [21], to improve bounding box localization. It avoids reliance on the classification confidence score by assigning a localisation score to each box, which like [22], is used to select the best box. The localisation score is however not sufficient to select the best box, with [21] relying on application of G-NMS or S-NMS followed by mean averaging of localization confidence scores of clustered boxes, and calculation of standard deviation to measure the uncertainty of the estimated bounding box location. Although [21] improved AP on MS COCO, its usage requires retraining of the object detector integrating the new loss function. It is also significantly more computationally expensive, and it only addresses the issue of poor localisation, and does not improve recall. Conversely, Confluence resolves both issues, by treating the strong coherence of bounding boxes as a measure of localization confidence without complex, computationally expensive calculations, retraining of networks or need of a localization loss.
Another limitation of NMS is its high sensitivity to the IoU threshold [37], [36], [35], which makes it difficult to achieve an optimal balance between retention of true positives and removal of false positives. This is particularly true for highly occluded settings such as crowds and has resulted in the development of crowd setting specific modifications of NMS. Rather than avoiding the use of IoU, these modifications are premised on attempting to minimise the limitations of IoU in adequately representing true and false positives.
Adaptive NMS [36] proposes a dynamic per instance IoU threshold to achieve a better balance between retention of true positives and elimination of false positives, tailored to pedestrian detection. A learnable subnetwork determines the IoU threshold, which changes depending on the density of the image. It achieved competitive results on the CityPersons and CrowdHuman benchmarks. Alternatively, handcrafted image descriptors of person silhouettes are exploited by [35] to rescore candidate detections to improve performance of NMS on the PETS [76], COCO Person [56] and Okutama-Action [77] datasets. However, [36] and [35] are significantly more computationally expensive than traditional NMS and not evaluated in general object detection tasks.
Similarly [78] proposes attribute-aware NMS, which leverages semantic attributes such as density and diversity gained from a pedestrian-oriented attribute map to minimise false positives occasioned from using a high IoU threshold during NMS. Although it outperformed G-NMS, its reliance on a custom network designed specifically for crowd detection means it cannot be easily adopted in standard state-of-the-art object detection pipelines.
An adaptive threshold for NMS in video sequences is proposed by [71]. It uses high confidence bounding boxes in key frames to improve detection in lower scoring frames. Although gains in AP were achieved, it is not useful for standard image-based object detection due to its reliance on multiple consecutive frames for threshold adaptation. In contrast, Confluence can be used in both image object detection studies and video-based object detection without retraining or additional image information, achieving significant gains in performance.

METHODOLOGY
Confluence derives its name from the highly confluent, aggregated clusters of bounding boxes returned by a neural network when an object is detected. Rather than treating the excessive proposals as problematic, Confluence embraces them as a way of identifying and retaining the bounding box which best represents the object location. The clustering can be interpreted as a collective vote on object location, where the box that best represents every other box is the optimal box. Bounding box confluence is also an effective way of removing those false positives that are confluent with the retained box.
Confluence is a recursive, two-staged algorithm which first retains an optimal bounding box, and then removes false positives that are confluent with it. Retention is achieved using a confidence weighted Manhattan Distance inspired proximity measure to evaluate bounding box coherence, enabling retention of the bounding box that best represents all boxes in a cluster. The second stage involves removal of all bounding boxes which are confluent with the retained bounding box. This process is repeated until all boxes have been processed.

Manhattan Distance
The Manhattan Distance (MD) or L1 norm, is the sum of the vertical and horizontal distances between two points [55]. The MD between the points = ( , ) and = ( , ) is shown by equation (1): Each bounding box can be represented by two diagonally opposite corners. For example, = ( , ) can be defined by the upper left corner = ( , ) and the lower right corner = ( , ).
We propose a proximity measure ( , ) between any two bounding boxes = ( , ) and = ( , ), represented by the sum of the MD between the upper left corners 1 = ( 1 , 1 ) and 2 = ( 2 , 2 ), and the lower right corners 1 = ( 1 , 1 ) and 2 = ( 2 , 2 ) of the two boxes as given by equation (2): A small ( , ) value denotes highly confluent boxes and , whilst a high ( , ) value indicates boxes that are not attributable to the same object -they may be somewhat overlapping, or completely disjoint. A diagrammatic representation of ( , ) is given in Fig 3. Let ( ) be a set of all boxes bounding the same object as box , such that ( ) does not include itself. We define proximity ( ) of a box as the mean value of the proximities of the box to all the boxes in ( ): A bounding box surrounded by a dense cluster of bounding boxes will be characterized by very low ( ) values, in comparison to a bounding box which is loosely surrounded by bounding boxes. The latter could be correctly categorized as an outlier, or as suboptimal. In effect, this provides a measure of the object detector's confidence in the presence of an object at a given location. On this basis, we propose that that the bounding box with the lowest ( ), value represents the most confident detection for a given object.
Notably, this concept overcomes an issue faced by all variants that rely on the classification confidence score. In situations where the highest scoring bounding box is sub- optimal in comparison with another lower scoring bounding box, NMS returns the sub-optimal bounding box, as illustrated by Fig 1. In contrast, the ( ) measure allows for the bounding box that is most confluent with all other bounding boxes assigned to a given object to be favored.

Normalization
The concept outlined in the previous section operates effectively in circumstances where bounding boxes are of similar size. However, in practice, objects and their corresponding bounding boxes will be of varying sizes. This poses a problem when regulating bounding box retention or removal using a hyper-parameter based on ( ). This is because a trade-off between removing large false positives and retaining small true positives would need to be reached. For example, Fig. 4 shows two large bounding boxes on the right, which denote the same object. The two small bounding boxes on the left denote two separate objects. However, when the proximity calculation is performed on each pair of boxes, the same value is obtained, as follows: This poses the problem of distinguishing between bounding boxes belonging to the same or distinct objects, particularly when there are significant differences in size.
To overcome this issue, a normalization algorithm was used to scale the bounding box coordinates between 0 and 1, whilst preserving their relationship with each other. The normalization algorithm transforms each coordinate of the boxes , and as follows.

Intra-cluster Retention and Removal
As all bounding box pairs are normalized between 0 and 1, any pair of intersecting bounding boxes ( , ) will have a ( , ) value below 2. However, empirical observation suggests that most clusters will be characterized by proximity values between 0 and 1.
In Fig. 5 and Fig. 6, each point on the horizontal axis represents a bounding box , 1 ≤ ≤ , while the vertical axis represents the proximity ( 0 , ) between a randomly selected box 0 , and box . To visualize the relationship between boxes, they were ordered so that ( 0 , 1 ) ≤ ( 0 , 2 ) ≤ ⋯ ≤ ( 0 , ). This reveals the blob-like nature of ( , ) clustering, where each horizontal blob represents an object. ( , ) values generally lie between 0 and 1, with the optimal bounding box being represented within the flattest gradient of the blobs.
Thus, if the ( , ) value of any two bounding boxes is below the user defined Confluence threshold (Ct), it is assumed that they belong to the same cluster, and therefore refer to the same object, or to one or more high density objects. The optimal intra-cluster bounding box is found, by calculating the mean ( ) value, using equation (3), of each box in the cluster. The bounding box with the lowest ( ) is the most confluent and is retained.
All bounding boxes that are confluent with this chosen box are likely to be false positives. Thus, their classification confidence scores are either removed or decayed as a function of their confluence with the chosen box.

Confidence Score Weighting
The majority of NMS variants, including G-NMS and S-NMS use a single classification confidence score returned by the object detector as the sole means by which an 'optimal' bounding box is selected. In contrast, Confluence assesses the optimality of a given bounding box by comparing both its confidence score and its ( ) values with competing bounding boxes. To achieve this, the ( ) value is weighted by the classification confidence score , as follows: As is a value which lies between 0.01 and 1 (all classification confidence scores lie between 1-100%), this in effect provides a bias in favour of high confidence boxes by 1 github.com/ashep29/confluence artificially reducing the value of ( ) (Note that all bounding boxes with confidence scores below 0.01 are not considered). Conversely, the ( ) value of low confidence boxes will be greater. This increases the likelihood of a high confidence box being selected, as bounding boxes are chosen based on small ( ) values.
This algorithm is based on the principle that a powerful classifier can be constructed by using the sum of weaker individual classifiers [79], [80]. Each individual ( ) value is a weak classifier on its own, but when these weak classifiers are collectively interpreted, they provide a powerful means to classify a bounding box as either confident -via high confluence, or not confident, via disparate positioning with respect to other bounding boxes. In essence, this provides a vote of confidence by the object detector on which bounding box best represents every other bounding box assigned to an object. Our experimental results presented in Tables 1-3 suggest that this is a reliable means to accurately identify true positives, whilst effectively minimizing false positives. This allows achievement of optimal precision and recall values.

Implementation
The pseudocode outlining the Confluence is provided by Algorithm 1. Both Confluence and C-NMS were implemented in Python and are freely available on GitHub. 1

3.
is the optimal bounding box, which is added to the final detections set D and removed from the dictionary I (line 19). 4. Next, remove all bounding boxes that have been identified as neighboring as these are treated as locating the same object. 5. Repeat steps 2-4 until all boxes have been processed.
The computational complexity of each step of Confluence is O(n), where the size of the input set of bounding boxes is n. This is due to the calculation of the normalized proximity score. As this measure is computed for each bounding box against every other bounding box, Confluence has an overall computational expense of O(n 2 ).
Although the computational expense of Confluence is not significant due to the recursive reduction in the size of the set of bounding boxes, it may still be suboptimal for applications that prioritize speed. Thus, we also propose the non-IoU NMS algorithm Confluence-NMS (C-NMS).
Like other NMS variants, C-NMS retains the bounding box with the maxima score, however its performance in minimizing false positives whilst maximizing true positives is significantly improved. This is because it relies on Confluence to suppress false positives and retain true positives, improving both recall and precision. Algorithm 2 provides pseudo-code illustrating C-NMS. C-NMS operates as follows: 1. In a dictionary I, mapping bounding boxes to their corresponding confidence scores, find the box with the highest score (line 3).

If ( , ) is less than or equal to the user defined
Confluence threshold Ct (line 8) remove box from the dictionary (line 9). 5. Repeat this process until all boxes have been retained in the set D and/or suppressed. Both Confluence and C-NMS have been implemented such that bounding box suppression can be achieved linearly or using Gaussian weighting. This is inspired by the use of Gaussian weighting by [7] to improve recall by down-sampling rather than suppressing high-scoring bounding boxes that overlap with optimal boxes.

DATASETS, ALGORITHMS AND EVALUATION
Experimental results presented in this paper were collected on the publicly available 2017 MS-COCO mini validation (mini-val) dataset [56] and 2018 CrowdHuman [57] validation datasets. These datasets were chosen to demonstrate Confluence on widely recognized standard and high-density, high-occlusion benchmarks.
The COCO mini-val set contains 5000 images of 80 classes. The CrowdHuman validation set contains 4370 images of people in highly crowded settings, with an average of 23 humans per image [57]. The AP calculations were obtained using the COCO-style evaluation metrics via the standard COCO API, using default settings including 100 maximum detections per image. 2 github.com/bharatsingh430/soft-nms 3 github.com/hasanirtiza/Pedestron The G-NMS and S-NMS algorithms used were implemented by [21] and are publicly available on their GitHub repository. 2 These algorithms were evaluated using two state-of-the-art object detectors; RetinaNet-ResNet50 [5], and Mask-RCNN (HRNet) [3]. We used a Mask R-CNN implementation (and associated pretrained model) published by [81], which is publicly available on GitHub. 3 The Mask-RCNN model used was trained by [81] on the CrowdHuman training set as described in [81]. The RetinaNet implementation and pretrained model used was obtained from GitHub. 4 It was trained on the MS COCO 2017 dataset as outlined on the repository. We did not conduct further model training, instead selecting publicly available pre-trained models in all cases. We simply replaced the default G-NMS module with the S-NMS implementation provided by [21], and our implementations of Confluence and C-NMS, gathering all results using default settings.
Threshold sensitivity analysis was also conducted to demonstrate the robustness of the Confluence threshold (Ct), by examining changes in AP attained by G-NMS and S-NMS over variations in IoU threshold in comparison to change in AP achieved by Confluence across variations in the Ct threshold. Sensitivity analysis was achieved using RetinaNet, applied to the MS-COCO mini-val dataset. 4 github.com/fizyr/keras-retinanet   AP results on 20 randomly selected COCO classes. C-NMS outperforms G-NMS and S-NMS strongly for classes that are often present in highly occluded or dense images, whereas performance on images characterised by lack of occlusion, low-density or lone objects is similar to NMS.

RESULTS
In this section, we provide performance results on both MS-COCO mini-val and CrowdHuman datasets. In Table  1 we compare the performance of Confluence and C-NMS against G-NMS and S-NMS on the MS-COCO mini-val dataset using RetinaNet. The Confluence Ct threshold was set to 0.7. It is clear that both Confluence and C-NMS improve object detector performance, particularly on the more stringent AP@0. Significant improvements in AR are also achieved by Confluence and C-NMS, with gains of 1.5-5.3% at max detections per image of 100. Confluence and C-NMS also outperform G-NMS and S-NMS-L across all object sizes by up to 3.5% AP and 8.1% AR.
Gaussian confidence score decaying is more computationally expensive, and only results in gains in AP and AR when a very low confidence threshold is used, reducing its practical applicability. Regardless, we provide results for C-NMS-G, which outperforms S-NMS-G by 0.4% on the AP@0.5-0.95 metric and 0.2% on the PASCAL VOC 0.5% AP metric. Gains in AR range from 1.3% on the max detections of 100 metric, and gains of up to 1.6% on objects of varying sizes. These improvements are significant for the MS-COCO dataset and evaluation metric. Table 2 presents RetinaNet performance across 20 randomly selected classes in the MS-COCO dataset. Confluence and C-NMS outperform G-NMS and S-NMS strongly for classes that are often present in highly occluded or dense images, such as person, truck, zebra, and banana, where improvements ranged from 2.51-9.67%. Gains in AP for object detection on classes that are often represented as lone objects and not occluded, for example laptop, dog, cat, and airplane, are insignificant and often equivalent to AP achieved by G-NMS and S-NMS. Table 3 presents the same experiments performed using Mask-RCNN evaluated on the CrowdHuman dataset. On the AP@0.5:0.95 metric, Confluence achieved gains in AP of 1.2-4%, while C-NMS outperformed G-NMS and S-NMS-L by 1.0-3.8%. Similarly, at the PASCAL VOC AP metric of IoU@0.5, Confluence improves AP by 1.4-6% while C-NMS outperforms G-NMS and S-NMS-L by 1.2-5.8%. Significant gains in AR are also achieved with gains of 2.4-7.3% at max detections per image of 100. Confluence and C-NMS also outperforms G-NMS and S-NMS-L across all object sizes by up to 4.4% AP and 7.4% AR. C-NMS-G significantly outperforms S-NMS-G by 2.9% on the AP@0.5-0.95 metric and 2.2% on the PASCAL VOC 0.5% AP metric. AR improves by 2.0% on the max detections of 100 metric, and up to 2.6% on objects of varying sizes.
Notably, improvements in AP and AR were greater on CrowdHuman and MS-COCO classes that frequently involve high occlusion. This suggests that the task of bounding box retention and removal can better be achieved by interpreting confluent bounding boxes according to coherence of borders rather than overlap.

Sensitivity Analysis
This subsection presents the results of threshold sensitivity analysis comparing the sensitivity of the G-NMS and S-NMS IoU threshold to the Confluence and C-NMS Ct threshold. High sensitivity is detrimental due to the consequently high fluctuations in AP and AR when the algorithm is confronted with variations in image density and occlusion within a dataset. The results shown in Fig. 8 were collected using RetinaNet and the MS-COCO minival dataset, evaluated using the AP@0.5:0.95 metric.
Both parameters were varied incrementally by 0.1. The optimal threshold for G-NMS and S-NMS on MS-COCO lies between 0.3-0.6. This is because a high IoU threshold results in only highly overlapping bounding boxes being removed, resulting in a high number of false positives but  greater recall. In contrast, a lower IoU threshold removes more bounding boxes, reducing recall yet minimising false positives.
The optimal range for Confluence and C-NMS ranges between 0.5 and 0.8. This is because a high Ct value indicates low proximity of bounding box borders, while lower values indicate greater bounding box confluence. Thus, the higher the Ct threshold, the more bounding boxes are removed. This is why the optimum threshold value for Confluence and C-NMS is higher than that used by the IoU dependent G-NMS and S-NMS.
Performance by both algorithms tends to decrease outside these ranges. Note that the variation in AP for Confluence and C-NMS within its optimal range is more stable than G-NMS and S-NMS, with variation in AP for Ct being 0.3% while variation in AP for IoU is 0.6%. This means that the Ct threshold is less sensitive to fluctuations in object density and occlusion, which makes it more robust. Furthermore, as shown by Fig. 8 performance of Confluence and C-NMS always remains approximately 1.5% better than G-NMS and S-NMS, even at the optimal IoU threshold.

DISCUSSION
This section will relate the quantitative results presented in Section 5 to a qualitative comparison of Confluence, C-NMS and the IoU-based G-NMS and S-NMS. It will explain how and why Confluence returns optimal bounding boxes, using qualitative data to demonstrate why coherence of bounding box borders is a more appropriate metric than IoU to use in bounding box selection and suppression in object detection. We will also provide insight into possible future work to further improve the performance and applicability of the Confluence algorithm.

Qualitative Comparison of Confluence with IoU-based G-NMS and S-NMS
A fundamental problem with IoU based suppression of bounding boxes is the elimination of true positives in high density images. Once the highest confidence bounding box b is selected, any detection with a sufficiently high overlap with b is removed. In situations where objects are occluded by other objects of the same class, for example, when a person occludes another person as shown in Fig. 1 and 9, high IoU will often result in the suppression of detections denoting true positives. The raw, unfiltered object detector output is illustrated in Fig. 1 and 9 at a confidence threshold of 5%. It is evident by the thick confluence of proposals that all objects are detected and localized correctly. Thus, the aim of the postprocessing stage is to maximize precision by selecting an optimal detection to represent each true positive, without lowering recall by suppressing true positives. IoU variants of NMS, such as G-NMS and S-NMS do not achieve this when applied to these images. Their reliance on the maxima confidence score causes them to return suboptimal bounding boxes, while their IoU dependency causes them to suppress true positives. In contrast, Confluence uses the heavy cluster of bounding boxes as an indicator of the presence of an object, thus returning one bounding box per cluster. This results in both higher recall and precision.
Unlike G-NMS and S-NMS, Confluence rewards high confluence, interpreting clustering as a vote of confidence by the neural network on the likeliest location of an object in an image. Thus, perhaps the most effective means by which Confluence can be evaluated is by qualitative data. Fig. 9, 10, 11, 12, 13 and 14 illustrate qualitative results using the output of RetinaNet on images from the MS-COCO dataset.  1) Improved Bounding Box Selection: As previously discussed, object detectors return many bounding boxes, and their corresponding scores in locations where the probability of an object being present is high. Although most NMS variants interpret the bounding box with the highest confidence score within a cluster as the optimal bounding box, there are many instances where the highest scoring box is not the optimal bounding box. A few examples of these situations are provided in Fig. 10 and 11.
Many of these instances involved people, at close proximity, such as those shown in Fig. 11. G-NMS and S-NMS simply return the highest scoring box, even if it is too large or too small. In contrast, Confluence has the capacity to return a more accurate box by taking advantage of the confluence of tighter fitting bounding boxes around each person.
Rather than simply selecting the highest scoring bounding box, Confluence selects the bounding box which is the most coherent with every other bounding box in the cluster. Consequently, if a highly confluent bounding box is a better representation of the object, it will be returned by Confluence despite having a lower confidence score. This improvement in bounding box selection is most evident when the object detector is a proposal based DCNN, such as RetinaNet, which has a tendency of returning very dense confluence around objects.
2) Improved Bounding Box Selection Improves Recall: The reliance of NMS on the classification confidence score as the sole means by which an optimal bounding box is selected reduces recall. For example, Fig. 12 illustrates how G-NMS selects the highest confidence box (87.7% confidence, shown in red) to locate the boy, but this bounding box is sub-optimal in comparison to the bounding box selected by Confluence (82.1% confidence). Due to G-NMS's selection of the maxima, it suppressed the bounding box allocated to the man standing behind the boy (shown in yellow), as they share a high IoU, thus reducing recall. In contrast, Confluence retains both bounding boxes locating both the man and boy. Fig. 10. Even when objects are not occluded, or highly proximate, the highest scoring box is not always optimal. This is true for all classes.   11. The optimal box does not always have the highest confidence score. Often, a bounding box which is too large is returned in instances where objects, such as people are at close proximity.

3) Suppression of False Positives via Manhattan Distance
Improves Accuracy: It was observed that due to suppression of false positives via IoU, in some situations, G-NMS suppresses a bounding box which has a high IoU with a higher confidence box, even if it correctly locates a second object. NMS is then forced to select a sub-optimal bounding box to locate the second object, due to suppression of the optimal box. For example, Fig. 13 shows the output of G-NMS, and Confluence on the same image. G-NMS suppresses the optimal, high confidence (66%) box allocated to the giraffe in the background due to its high IoU with the high confidence (94.8%) box allocated to the giraffe in the foreground. It is then forced to return a low confidence (50.7%), sub-optimal bounding box for the giraffe in the background.
This issue can be rectified by increasing the NMS IoU overlap threshold, however this comes at a cost -the number of false positives returned increases significantly. In contrast, the Confluence algorithm uses areas of confluence returned by the object detector to determine whether or not a second object is present, thus making it more robust to this issue. It has the capacity to return optimal bounding boxes without returning false positives. 4) Improved Recall: Confluence achieves better recall than G-NMS because bounding box removal is based on the coherence of bounding box borders with each other, rather than the extent of their overlap. The benefits of this approach are most evident when objects are occluding each other, for example, when a person is standing in front of another person, as shown in Fig. 1, 8 and 12. Although the smaller bounding box is a subset of the larger box, its borders are not sufficiently coherent to be removed by Confluence. However, they share a large overlap, which means the lower confidence box is removed. This suggests that Confluence is more robust to high occlusion and explains why its recall is higher than G-NMS on all tested object detectors.
One shortcoming of the use of Confluence to remove false positives is its tendency to retain those false positives which are not confluent with other boxes in a given cluster. This can be seen in Fig. 14. The tennis racket is surrounded by two bounding boxes rather than one (see annotation 1), as the two boxes are not confluent. In situations like this, the object detector is not confident, which causes it to return spurious bounding boxes around an object. In these (uncommon) cases, NMS has higher precision, as it harshly removes any highly overlapping bounding boxes.

Future Work
Our experimental results strongly indicate that Confluence and Confluence NMS are superior alternatives to IoU-based variants of NMS including the de-facto standard Greedy NMS and the state-of-the-art Soft NMS in both multi-class and single-class object detection. However, the extent of its improvement over G-NMS and S-NMS may have been obscured by the way that the MS-COCO AP calculation processes bounding boxes. In AP calculators, bounding boxes are ranked by confidence score [43], [44], [9]. Consequently, if a sub-optimal box has Fig. 13. The optimal bounding box for the background giraffe on the far left (yellow) is suppressed due to IoU, which forces G-NMS to return a sub-optimal bounding box.
Fig. 14. The giraffe numbered 4 is a subset of the bounding box allocated to the giraffe in the background. It is removed by NMS, yet retained by Confluence, accounting for its higher recall. This also occurs in crowds, as shown by 2, and where objects are occluding each other, as shown by 3. a higher confidence score than a superior lower confidence box, the sub-optimal bounding box will be chosen by the AP calculator, and the superior box chosen by Confluence will be ignored. This degrades the overall AP score. Thus, future work could encompass the development of a confluence score to be used instead of the confidence score to rank bounding boxes in AP calculations. This would overcome the disadvantage faced by Confluence, enabling more effective evaluation of the extent of improvements in both bounding box retention and removal.
Furthermore, the hyper-parameter used by Confluence could be learned to optimize performance in any object detector. Another interesting area of research could be investigation of the applicability of the Manhattan Distance principles used by Confluence as a possible non-IoU alternative to Jaccard Index based regression modules, and AP calculators.

CONCLUSION
We have proposed Confluence, a non-IoU alternative to Non-Maxima Suppression and its variants. Confluence does not rely on IoU or the maxima classification confidence score in bounding box retention and suppression. Empirical results obtained using multiple object detectors on the MS COCO and CrowdHuman benchmarks attest to its superior performance when compared to the de-facto standard and state-of-the-art NMS variants.
Furthermore, threshold sensitivity analysis indicates that Confluence is more robust to occlusion. Finally, it can be seamlessly integrated within currently used object detectors without modifications or training, making it an attractive alternative to NMS variants in object detection. Confluence represents a paradigm shift away from the heavily researched and used IoU towards a more robust boundary distance metric, which may be used to replace IoU in other applications, ranging from regression modules to localization losses and AP calculators.