{'fraction_non_alphanumeric': 0.03686504436257977, 'fraction_numerical': 0.008431484460125564, 'mean_word_length': 4.634702528870048, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Revisiting 3D Object Detection From an Egocentric Perspective

3D object detection is a key module for safety-critical robotics applications such as autonomous driving. For these applications, we care most about how the detections affect the ego-agent's behavior and safety (the egocentric perspective). Intuitively, we seek more accurate descriptions of object geometry when it's more likely to interfere with the ego-agent's motion trajectory. However, current detection metrics, based on box Intersection-over-Union (IoU), are object-centric and aren't designed to capture the spatio-temporal relationship between objects and the ego-agent. To address this issue, we propose a new egocentric measure to evaluate 3D object detection, namely Support Distance Error (SDE). Our analysis based on SDE reveals that the egocentric detection quality is bounded by the coarse geometry of the bounding boxes. Given the insight that SDE would benefit from more accurate geometry descriptions, we propose to represent objects as amodal contours, specifically amodal star-shaped polygons, and devise a simple model, StarPoly, to predict such contours. Our experiments on the large-scale Waymo Open Dataset show that SDE better reflects the impact of detection quality on the ego-agent's safety compared to IoU; and the estimated contours from StarPoly consistently improve the egocentric detection quality over recent 3D object detectors.


Introduction
3D object detection is a key problem in robotics, including popular applications such as autonomous driving. Common evaluation metrics for this problem, e.g. mean Average Precision (mAP) based on box Intersection-over-Union (IoU), follow an object-centric approach, where errors on different objects are computed and aggregated without taking their spatiotemporal relationships with the egoagent into account. While these metrics provide a good proxy for downstream performance in general scene understanding applications, they have limitations for egocentric applications, e.g. autonomous driving, where detections are used to assist navigation of the ego-agent. In these applications, detecting potential collisions on the ego-agent's trajectory is critical. Accordingly, evaluation metrics should focus more on the objects closer to the planned trajectory and to the parts/boundaries of those objects that are closer to the trajectory.
paper, we take a novel approach to 3D object detection from an egocentric perspective. We start by reviewing the first principle: the detection quality relevant to the ego-agent's planned trajectory, both at the moment and in the future, has the most profound impact on the ability to facilitate navigation. This leads us to transform detection predictions into two types of distance estimates relative to the ego-agent's trajectory -lateral distance and longitudinal distance (Fig. 1). The errors on these two distances form our support distance error (SDE) concept, where the components can either be aggregated as the max distance estimation error or used independently, for different purposes. These two types of support distance measure how far an object's shape boundary is to the observer (egoagent) in both the direction along the observer velocity (longitudinal) and perpendicular to it (lateral).
Compared to IoU, SDE (as a shape metric) is conditioned on the spatio-temporal relationship between the object and the ego-agent. Even a small mistake in detection near the ego-agent's planned trajectory can incur a high SDE (as in Fig. 2 left, object 3). Additionally, SDE can be extended to evaluate the impact of detections to the ego-agent's future plans (for cases where an object comes close to the planned trajectory later in time). This is not feasible for IoU, which is invariant to the ego-agent position or trajectory(shown in Fig. 2).
Using SDE to analyze a state-of-the-art detector [44], we observe a significant error discrepancy between using a rectangular-shaped box approximation and the actual object's boundary, suggesting the need for a better representation to describe the fine-grained geometry of objects. To this end, we propose a simple lightweight refinement to box-based detectors named StarPoly. Based on a detection box, StarPoly predicts an amodal contour around the object, as a star-shaped polygon.
Moreover, we incorporate SDE into the standard average precision (AP) metric and derive an SDE-based AP (SDE-AP) for conveniently evaluating existing detectors. In order to make an even more egocentric AP metric, we further add inverse distance weighting to the examples, obtaining SDE-APD (D for distance weighted). With the proposed metrics, we observe different behaviors among several popular detectors [41,44,67,20] compared to what IoU-AP would reveal. For example, PointPillars [20] excels on SDE-AP in the near range in spite of its less competitive overall performance. Finally, we show that StarPoly consistently improves upon the box representation of shape based on our egocentric metric, SDE-APD.
Egocentric Computer Vision Egocentric vision has been studied in various applications. To name a few, understanding human actions from egocentric cameras, including action/activity recognition [9,35,28,49,36,51,15,52], action anticipation [43,16], and human object interaction [26] have been widely studied. Egocentric hand detection/segmentation [23,22,1,42], and pose estimation [54,66,31] are among other applications. Arguably, 3D detection for autonomous driving can be naturally viewed as another egocentric application where data is captured by sensors attached to the car. However, classic IoU-based evaluation metrics ignore the egocentric nature of this application.
3D Object Detection Metrics Various extensions to the average precision (AP) metric have recently been proposed for the autonomous driving domain. nuScenes [2] consolidated mAP with more finegrained error types. Waymo Open Dataset [53] introduced mAP weighted by heading (mAPH) to reflect the importance of accurate heading prediction in motion forecasting. [34] proposed to examine Figure 2: Illustration of IoUs and lateral distances in a real scene. We visualize the scene from a bird's eye view where Lidar points are gray; green boxes are ground truth boxes; and red boxes are detector boxes. Left: We show that IoU as an object-centric measure is not directly reflecting the risk of collision (colored in blue) -the high risk mistake of the object 3's box is not reflected by the high IoU. In contrast, while object 2 has a lower IoU, its box boundary is accurately estimated, thus the impact to ego-agent planning is limited. As shown, compared to IoU, SDE is more indicative of the perception quality's impact on driving. Right: We show how SDE changes when evaluated at a future time (colored in purple), reflecting how the current frame's perception quality influences decision making into the future. The detection box is transformed to a future frame based on the rigid motion between the ground truth boxes at T = 0s and T = 1s (which excludes the error introduced by motion prediction). While object 1 has low SDE lat at T = 0s on the left, its error significantly increases at T = 1s, as the box cannot capture the fine-grained geometry at the object corner (see the zoom in view).  For each evaluation moments from the prediction time to 10s in the future, we compute the CDA, mean IoU (mIoU), and mean SDE (mSDE). We see from the curve that mIoU is not correlated with the accuracy drop as IoUs don't vary with ego motions; while mSDE is inversely correlated to collision detection accuracy due to its egocentric nature. detection quality from the planner's perspective, by measuring the KL-divergence between future predictions conditioned on either noisy perception or ground truth. However, factors such as different planning algorithms or model training setups may cause this approach to yield inconsistent outcomes.

Measure
Boundary-based Segmentation Metrics A different class of shape metrics on semantic segmentation masks evaluates the match quality of ground truth and predicted segmentation boundaries.

An Egocentric Shape Metric: Support Distance Error
Understanding the quality of modern 3D object detectors from an egocentric perspective is an underexplored topic and is open for new egocentric shape measures. In this section, we first look at the limitations of the box Intersection-over-Union (IoU) measure, the de facto choice to evaluate detection quality in popular benchmarks [11,53,2] and then introduce our newly propose egocentric shape metric: support distance error (SDE).

Limitations of box-based IoU
IoU is an object-centric measure based on volumes (or areas). As illustrated in Fig. 2 (left), a prediction box with a relatively high IoU can still exhibit a high risk for an ego-agent (the protruding box can cause the planner to brake suddenly, which in turn could lead to a tailgating collision).
To understand such behavior at scale, we use collision detection as a "gold standard" to quantitatively reveal the limitation of IoUs. We select all the collisions reported by either the ground truth or a state-of-the-art PV-RCNN detector [44] in the validation set of the Waymo Open Dataset [53]. A ground truth collision is defined as an event where the object shape (approximated by the aggregated object LiDAR points across all of its observations) overlaps with the extended ego-agent shape (approximated by a bounding box of the ego-agent, scaled up by 80%). Collisions are estimated using detector boxes as the object's shape. Table 1 presents the mean and median IoUs for true positive and false positive collision detections, whose difference is minimal, indicating that IoU is not effectively reflecting collision risk.

Support Distance Error (SDE)
In autonomous driving, one of the core uses of detection is to provide accurate object distance and shape estimates for motion planning (which has collision avoidance as a one of the primary objectives). Instead of using box IoU, we can measure distances from the estimated shapes to the ego-agent's planned trajectory. Specifically, we propose two types of distance measurements ( Fig. 1) 2 : • Lateral distance to an object: The minimal distance from any point on the object boundary to the line in the ego-agent's heading direction. This distance is critical for the ego-agent to plan lateral trajectory maneuvers. • Longitudinal distance to an object: The minimal distance from any point on the object boundary to the central line perpendicular to the ego-agent's heading direction. This distance is important to determine the speed profile and keep a safe distance from the objects in front.
We use the term support distances for these two distance types, as they "support" the decision making in trajectory planning, and name the error between the ground truth support distance and the one estimated from a detector's output as the support distance error (SDE). We use SDE lat to denote the lateral distance error and SDE lon for the longitudinal error, and we define SDE as the maximum of the two. This formulation leads to two conceptual changes compared to IoU: we shift our focus from volume to boundary and from object-centric to ego-centric.
This definition can also be extended to measure the impact of the detection quality on future collision risks. If we compute distances from the object boundary to the tangent lines at a future position (at time t) on the ego-agent's trajectory, we can compute SDE for different future time steps (denoted as SDE@t). This is equivalent to measuring how close the object is to a future location of the ego-agent.
To make the definition concrete, at time T = t, we assume the ego-agent's pose is e (t) = (x (t) , θ (t) ), with x (t) ∈ R 3 as its center (e.g. the center of the ego-agent's bounding box) and θ (t) as its heading direction (e.g. clock-wise rotating angle around the up-axis). We define the "lateral line", the line crossing the ego-agent's center and in the direction of its heading, as l (t) lat ; and the "longitudinal line" perpendicular to it as l (t) lon . On the other hand, we assume we have an object o and its predicted boundary is B(o) as a set of points on the boundary. The lateral/longitudinal distance of o at the current frame (T = 0) is defined as: where d computes the point-to-line distance. If the line passes through the object boundary the SDE would be 0. Assume B gt (o) is the object ground truth boundary, then the lateral/longitudinal support distance error is defined as: The SDE sign has a physical meaning: positive errors mean the predicted boundary is protruding while negative means that a part of the object is not covered by the predicted boundary. For simplicity, we take the absolute value of SDE lat and SDE lon by default and formally define SDE = max(|SDE lat |, |SDE lon |), an aggregated value of both errors.
To measure the impact of current frame detection quality on future plans, we define SDE@t, which computes the SDE of an object t seconds in the future. Given the ground truth rigid motion R (t) of the object from T = 0 to T = t, we can transform its predicted boundary at frame T = 0 to its future position. In this way, the error patterns of the boundary can be consistently propagated into a future frame (see Fig. 2 right for an example). The rigid motion can be derived between pairs of ground truth boxes of the object. We denote the transformed B(o) as B (t) (o) . Note that it is different from the object shape prediction at time T = t: we are still measuring the quality of the T = 0 prediction, but within a future egocentric context. The future support distance can be formally defined as: Similarly, we define SDE α @t as the difference in SD α @t between the ground truth and the predicted boundary, where α ∈ {lat, lon}. Then SDE@t = max(|SDE lat @t|, |SDE lon @t|). We use SDE and SDE@0s interchangeably unless otherwise noted.
Metric implementation details To faithfully compute the support distance, we aggregate object surface points (from Lidar) across all frames, during which the object is observed (which cover different viewpoints of the object) as a surrogate shape to the ground truth. This allows us to effectively compute distances to the boundary without requiring costly object shape annotations/modeling. By default, we use the real driving trajectory. The same implementation applies when one would like to evaluate SDE by providing an arbitrary set of intended trajectories (from a planner or simulation).
Comparing SDE with IoU In Fig. 2, we see SDE lat is a highly useful indicator to reflect collision risk (for object 3). In Tab. 1, we show that the mean and median SDE are sensitive shape measures and are inversely correlated with the collision risk. Naturally with larger t, SDE@t increases, since the detections are based only on sensor data from the current frame T = 0. Fig. 3 shows how SDE@t and IoU change when we evaluate them at different time steps.
Note that both SDE and SDE@t are defined based on distances to the object boundary (usually the part closer to the ego-agent). Clearly, better detection quality and boundary representation will result in an improved SDE metrics, which leads to the main idea of our next section.

Shape Representations and SDE
In this section, we use SDE to analyze detection quality in safety-critical scenarios and highlight the importance of the shape representation therein. We further propose a new amodal contour representation and a neural network model (StarPoly) for contour estimation and demonstrate it produces significant SDE improvements.

Qualitative Analysis of Bounding Box Failure Cases
To understand how detector boxes perform under the SDE measure, we select PV-RCNN [44], a topperforming single-frame point-cloud-based detector in popular autonomous driving benchmarks [11,53]), for our analysis. All analysis is based on the Waymo Open Dataset [53] validation set. Fig. 4 illustrates some representative failure cases among the detector boxes, with high SDE.
We find that even when a box aligns reasonably well with the ground truth it can still incur high SDE. By comparing the predicted detection against the point cloud inside in the box, we notice that rectangular boxes typically do not tightly surround the object boundary. In particular, the discrepancy between box corners and the actual object boundary contributes a considerable amount of SDE. This observation inspires us to seek more effective representations of the fine-grained object geometry.

Convex Visible Contours
An intuitive solution to obtain a tighter object shape fit is by leveraging the Lidar points. Specifically, one can extract all points within the detector box (after removing points on the ground) and compute their convex hull, as a convex visible contour (CVC). In contrast to amodal object shape, CVC is computed only from the visible Lidar points at the current frame. Fig. 4 provides some visualizations.
Tab. 2 shows how CVC compares with bounding boxes in SDE. Considering that CVC is heavily dependent on the quality of the box it resides in, we also evaluate CVC directly based on the ground truth boxes, which can be seen as the upper bound for CVC (col. 6). We see that at near range, CVC can significantly improve SDE compared to the detector boxes (col. 2 vs 3). However, its effectiveness degrades at longer ranges (col. 2 vs 3) and its performance is inferior to ground truth boxes (col. 5 vs 6). We hypothesize that this is because CVC is vulnerable to occlusions, clutter and object point cloud sparsity at longer ranges, which are ubiquitous phenomena in real world data. In Fig. 5, the analysis based on SDE@t confirms that CVC performs better than detector boxes at the current frame but generalizes poorly to longer time horizons. To improve it, we need a representation that provides good coverage of both the visible and the occluded object parts.

Amodal Contour Estimation with StarPoly
We propose to refine box-based detection with amodal contours, a polyline contour that covers the entire object shape (See Fig. 6 for an illustration). Our model, StarPoly, implements contours as star-shaped polygons and predicts amodal shape via a neural network 3 . It can be employed to refine predicted boxes for any off-the-shelf detectors.
Input The input to the StarPoly model is a normalized object point cloud. We crop the object point cloud from its (extended) detection box. The point cloud is canonicalized based on the center and the heading of the detection box, as well as scaled by a scaling factor, s, such that the length of the longest side of the predicted box becomes 1.
Parameterization As shown in Fig. 6, the star-shaped polygon is defined by a center point, h, and a list of vertices on its boundary, (v 1 , ..., v n ), where n is the total number of vertices determining the shape resolution. We assume h is the center of the predicted box and sort (v 1 , ..., v n ) in clockwise order so that connecting the vertices successively produces a polygon. We constrain v i to have only 1 degree of freedom by defining v i = c i d i , where ( d 1 , ..., d n ) is a list of unit vectors in predefined directions. Consequently, predicting a star-shaped polygon is equivalent to predicting (c 1 , ..., c n ), for which we employ a PointNet [40] model (see the supplementary material for details).
Optimization Since ground truth contours are not available in public datasets, directly training the regression of (c 1 , ..., c n ) is infeasible. We resort to a surrogate objective for supervision. The objective combines three intuitive goals, namely coverage, accuracy, and tightness. The coverage loss encourages the prediction to encompass all ground truth object points (aggregated points in the object bounding box from all frames in which the object appears, with ground points removed). Moreover, as the input point cloud already reveals part of the object boundary visible to the ego-agent, the accuracy loss requires the prediction to fit the visible boundary as tight as possible. On the other hand, the tightness loss minimizes the area of the predicted contour. The combination of these three goals leads to the reconstruction of contours without requiring ground truth contour supervision. More formally, the coverage loss L c , the accuracy loss L a , the tightness loss L t , and consequently the overall objective L for one ground truth point cloud X are defined as follows: where x is a point from X, γ is a weight parameter for L t , and × represents cross product. Note that in L c , l and r are selected so that d l and d r span a wedge shape containing x (as shown in Fig. 6). Intuitively, L c is computing the barycentric coordinates of x with regard to v l and v r within the triangle hv l v r and encouraging x to be on the same side as h regarding v l v r -the necessary and sufficient condition for x ∈ hv l v r . Similarly, L a is forcing the points on the visible boundary B to be on the predicted boundary as well. Meanwhile, L t is pulling all v i towards h.

Results
In Tab. 2 we see that updating the bounding box output of PV-RCNN to StarPoly contours significantly improves the mean SDE under all distance buckets (e.g. at 0-5m, it improves from 10.7cm to 8.6cm, which is around 20% error reduction). Similar improvements also appear on the ground truth boxes (col. 4-6). In Fig. 5, we also show how StarPoly improves on SDE@t. In all time steps, the StarPoly has lower SDE than both bounding boxes and visible contours, showing its advantage of getting the best of both worlds.

Egocentric Evaluation of 3D Object Detectors
In this section, we incorporate SDE into the standard average precision (AP) metric and evaluate various detectors and shape representations on the Waymo Open Dataset [53].

SDE-AP: Detection AP based on the SDE shape metric
To compare different detectors on their egocentric performance, we cannot just use the SDE measure, which does not consider false positive (FP) and false negative (FN) cases. Therefore, we propose to adapt the traditional IoU-based AP (IoU-AP) to an SDE-based one (SDE-AP). Specifically, we replace the classification criterion for true positives (TP) from an IoU to an SDE-based threshold and use SDE = 20cm as the threshold (see the Supplementary material for more on why we selected this number). In addition, we use SDE-based criterion (instead of an IoU-based one) to match predictions and ground truth.
where d is the Manhattan distance from the prediction shape center to the ego-agent center and β is a hyper parameter controlling how much we focus on the close-by objects (we set β = 3, see the supplementary for more details).
The inverse distance weighted precision and recall are defined as IDT P/(IDT P + IDF P ) and IDT P/IDG respectively, both remain within [0, 1]. The SDE-APD is the area under the PR-curve. Similar to SDE, which is defined both for the current frame and for future frames (SDE@t), the SDE-AP and SDE-APD metric also have future equivalents SDE-AP@t and SDE-APD@t that can evaluate the impact of current frame perception on future plans.  In this subsection, we compare a few representative point-cloudbased 3D object detectors on the SDE-AP and SDE-APD metrics. We study several popular detectors: PointPillars [20], a light-weight and simple detector widely used as a baseline; PV-RCNN [44], a state-of-the-art detector with a sophisticated feature encoding; MVF++ [67,41] (an improved version of the multi-view fusion detector), a recent top-performing detector; and finally 5F-MVF++ [41], an extended version of MVF++ taking point clouds from 5 consecutive frames as input, the most powerful among all. Fig. 7 shows the SDE-AP with distance breakdowns for all detectors and Table 3 shows the egocentric SDE-APD metric. An interesting observation from Fig. 7 about IoU-AP is that, while the four detectors have fairly close IoU-APs at close ranges (e.g. [0m, 5m]), we see significant gaps among them at longer ranges (e.g. [20m, 40m]). Since there are more objects at longer ranges, those long-range buckets typically dominate the overall IoU-AP. In contrast, the SDE-AP is consistently more discriminative of the detectors especially for the very short range   in [0m, 5m]. We even see some change of rankings -PointPillars, with the lowest overall IoU-AP, outperforms PV-RCNN and MVF++ in close-range [0m, 5m] SDE-AP, suggesting it has a particularly strong short-range performance. This also implies that simply examining IoU-AP for selecting detectors can be sub-optimal and our SDE-AP can provide an informative alternative perspective.

Comparing Various Shape Representations
In this subsection, we evaluate how detector output representations affect the overall detection performance in terms of SDE-APD evaluated at the current frame as well as into the future.
StarPoly implementation details. For the encoding neural network, we use the standard Point-Net [40] architecture followed by a fully-connected layer to transform latent features to (c 1 , ..., c n ).
We use a resolution n = 256 for all following experiments. ( d 1 , ..., d n ) is uniformly sampled from the boundary of a square. During training, γ and β are both set to 0.1, which is determined by a grid search over the hyperparameters. Please refer to the supplementary material for more model details.

Results
In Fig. 8, we compare the egocentric performance of different representations using SDE-APD. StarPoly consistently improves the egocentric result quality across the different detectors. Interestingly, StarPoly largely closes the gap between the different detectors, reducing the difference between 5F-MVF++ [41] and PV-RCNN by a factor of 3. This implies that StarPoly's amodal contours can greatly compensate for the limitations of the initial detection boxes, especially of those with poorer quality. StarPoly also outperforms convex visible contours (CVC) across all detectors. In Fig. 9, we evaluate the performance of the different representations at future time steps. We observe that StarPoly remains superior to the detector boxes across all time steps, differentiating it from the convex visible contours that decay catastrophically over time (shown in Fig. 5). Fig. 10 shows a scene with StarPoly amodal contours estimated for all vehicles. The zoom-in figures reveal how amodal contours have more accurate geometry, and lower SDE, than both boxes and visible contours. However, they are not yet perfect, especially on the occluded object sides. Improving contour estimation even further is a promising direction for future work.

Conclusion
In this paper, we propose egocentric metrics for 3D object detection, measuring its quality in the current time step, but also its effects on the ego agent's plans in future timesteps. Through analysis, we have shown that our egocentric metrics provide a valuable signal for robotic motion planning applications, compared to the standard box intersection-over-union criterion. Our metrics reveal that the coarse geometry of bounding boxes limits the egocentric prediction quality. To address this, we have proposed using amodal contours as a replacement to bounding boxes and introduced StarPoly, a simple method to predict them without direct supervision. Extensive evaluation on the Waymo Open Dataset demonstrates that StarPoly improves existing detectors consistently with respect to our egocentric metrics.

Supplementary Material
This document provides supplementary content to the main paper. In Sec. A, we expand the discussion on comparing our egocentric metric with a recent planner-based 3D object detection metric. In Sec. B, we provide more details of the StarPoly architecture and training. Sec. C explains more about how we select hyperparameters for our egocentric metrics. Finally, Sec. D shows more visualization results.

A Discussion
Similar to our metric, a recent work (planner-centric metrics) [34] also follows an egocentric approach to evaluate 3D object detection. It measures the KL-divergence of the planner's prediction based on either the ground truth or the detection. However, we would like to highlight two differences between our SDE-based metrics (SDE-AP and SDE-APD) and the planner-centric metrics: stability and interpretability.
Stability. In the planner-centric metrics [34], a pre-trained planner is required for the evaluation. Consequently, the metric is highly dependent on the architectural choices of the planner and may vary drastically when switched to a different one. Moreover, as the proposed planner is learned from data, many factors in the training can significantly affect the evaluation outcome: 1) the metric depends on a stochastic gradient descent (SGD) optimization to train the planner, which may fall into a local minimum; 2) the metric depends on a training set of trajectories, which will vary depending on the shift of data distribution. Furthermore, if the planner is trained on ground truth boxes, it may not reflect the preferences of a practical planner which is usually optimized for a certain perception stack. In contrast, SDE-based metrics don't require any parametric models. Its evaluation is consistent and can be universally interpreted across different datasets or downstream applications.
Interpetability. Because the KL-divergence employed in [34] only conveys the correlation of two sets of distributions, the magnitude of the metric is difficult to interpret. To fully understand the detection errors, one has to investigate the types of failures made by the planner, which vary depending on type of planner used. On the other hand, our proposed SDE directly measures the physical distance estimation error in meters. For SDE-based AP metrics, an intuitive interpretation is the frequency of detection, whose distance estimation error is within an empirically set threshold. Therefore, SDE-based metrics have a clear physical meaning, which translates the complex model predictions into safety-sensitive measurements.

B.1 Architecture
Our StarPoly model takes as input the point clouds cropped from (extended) detection bounding boxes. We apply a padding of 30cm along the length and width dimensions for for all detection boxes before the cropping. The point cloud is normalized before being fed into StarPoly based on the center, dimensions, and heading of each bounding box. In addition, the point cloud is subsampled to 2048 points before being processed by StarPoly. We use a PointNet [40] to encode the point cloud into a latent feature vector of 1024-d. Then, we reduce the dimensions of the latent feature vector from 1024-d to 512-d with a fully-connected layer. At last, another fully-connected layer is employed to predict the n-d parameters of a star-shaped polygon, where n is the resolution of the star-shaped polygon (as stated in Sec. 4.3). We use n = 256 for all the experiments in the main paper. As for selecting ( d 1 , ..., d n ), we uniformly sample directions on the boundary of a square, inspired by the prior that the objects of interest in this paper, i.e. vehicles, are symmetrical and are approximately of rounded square or rectangular shapes. Figure 11: SDE-APD with various distance thresholds. Note that at more stringent error threshold, e.g., 0.1m, SDE-APD clearly differentiates different detector's detected box quality, where 5F-MVF++ keeps outperforming others and PointPillars excels as well. Figure 12: SDE-APD with various β. We change the inverse distance weighting degree, β, in SDE-APD computation. Note that as we increase the degree, which means more focus is shifted to close objects, the SDE-APD of PointPillars [20] gradually catches up and even surpasses PV-RCNN [44] at β = 3. This is coherent with our study on SDE-AP's distance breakdowns. We can therefore conclude that β is a knob in SDE-APD to control the level of egocentricity based on object distances.

B.2 Optimization
We train StarPoly on the training split of the large-scale Waymo Open Dataset [53]. Because StarPoly aims to refine the results of a detector, we first use a pre-trained detector to crop out point clouds as described in Sec. B.1. Then we optimize StarPoly independently using the prepared point clouds. For all the experiments in the paper, we use StarPoly trained on MVF++. We find that StarPoly can generalize to different detectors even if trained only on one detector. For the StarPoly optimization, we use the Adam optimizer [17] with β 1 = 0.9, β 2 = 0.99 and learning rate = 0.001 an the parameters. For all experiments in the paper, we train StarPoly for 500, 000 steps with a batch size of 64 and set γ = 0.1.

C Details about SDE and SDE-APD C.1 Selection of metric hyperparameters
Selection of the SDE threshold As defined in main paper Sec. 5, we classify true positive predictions by comparing the SDE with a threshold. We use a threshold of 20cm for all experiments in the paper. Unlike the IoU threshold, our threshold has a direct physical meaning in safety-critical scenarios, i.e. the amount of estimation error by perception that an autonomous vehicle can handle. Therefore, it can be selected according to the real world use cases. In this paper, we select 20cm via analyzing the SDE of ground truth bounding boxes (as shown in main paper Table 2). We find the overall mean SDE of it to be 0.1m and therefore determine a relaxed value of 0.2m as the threshold. One can also use different thresholds for the evaluation as one can use different IoU thresholds for the box classification. Figure 11 illustrates the comparison among detectors with varying thresholds, i.e., 0.1m, 0.2m, 0.3m. We can see that PointPillars [20] demonstrates stronger performance compared to PV-RCNN [44] when the threshold is set more stringent. In addition, the effectiveness of using multi-frame information is more pronounced when the evaluation criterion becomes more rigorous.
Selection of β in the Inverse Distance Weighting To be more egocentric in our evaluation, we propose to extend the Average Precision (AP) computation by introducing inverse distance weighting. This strategy aims to automatically emphasize the objects close to the ego-agent's trajectory than those far away. As the number of objects grows roughly quadratically with regard to the distance, setting β = 2 (square inverse) would put equal weight for all distances. Since we want to highlight the importance of close-by objects, we go a further step and set β = 3. Fig. 12 shows the SDE-APD (evaluated at time step 0) with different choices of β. Setting β = 0 means all objects contribute equally to the AP metric, where see the greatest gap from the best and the worst detectors (5F-MVF++ [41] v.s. PointPillars [20]). As we increase the β, i.e. making the overall AP metric more egocentric, weighting more heavily on the close-by objects, we see the PointPillars (with great close-by accuracy) catches up with PV-RCNN [44] and MVF++. The general differences of different detectors also become smaller as they perform similarly well for objects close to the ego-agent's trajectory (the difference in the original IoU-AP is more related to their performance difference on far-away objects).   In our default definition, SDE is the maximum value of the lateral distance error and the longitudinal error. In Tab. 5 we investigate the composition of the two sub-distance-errors of the SDE. Specifically, we employ the detection boxes predicted by PV-RCNN as the detection output and calculate the mean and average of all valid SDE lat and SDE lon . Note that "valid" means that the object doesn't intersect with the lateral line (for SDE lat ) or the longitudinal line (for SDE lon ) and that the box is matched with a ground truth object. We also compute the portion of SDEs that are equal to its lateral component, i.e. SDE lat > SDE lon , and the portion of SDEs that are equal to the longitudinal component. From the statistics, we find that the lateral and longitudinal components contribute almost equally to the final SDE.

C.4 Distribution of signed SDE
In this work, we intend to bring attention to the idea of egocentric evaluation. We propose SDE without sign as a simple implementation of this idea with minimal hyperparameters required. It is straightforward to extend it to more complicated versions with the sign included. In Fig. 13, we provide a plotting of the distribution of signed SDE of detector boxes. It demonstrates that box predictions are generally oversized, i.e. with positive SDEs. Based on specific requirements of an   Table 6: Distributions of error measures in two types of collision detection cases. In "TP Collision", both the ground truth points and the prediction report a collision.
In "FP/FN Collision", either the ground truth (FN) or the prediction (FP) reports a collision. Here we use the aggregated point clouds to test collision. The results align with Tab. 1. application, one can also have more fine-grained thresholds, e.g. different thresholds for positive and negative, and select the most suitable set up based on their priorities.

C.5 Collision correlation of SDE and IoU based on contours
In Tab. 1, we use ground truth box to test collisions, to align with the evaluation of IoU. In Tab. 6, we re-computed the table using the contours drawn from our aggregated ground truth points, which should be the more accurate shape accessible. The gap between IoU and SDE is almost the same as the original Tab. 1 using boxes for collision tests.

D Qualitative Results
In this section we provide additional qualitative analysis. Fig. 14 shows how our metrics evaluate predictions at a future time step. We compare different representations both at the current time frame and at a future time frame. Our metrics are egocentric in the sense that they take into account the relative positions of the objects to the agent's trajectory in both the current and future time steps. Clearly, our proposed representation, StarPoly outperforms both box and convex visible contour (CVC) representations at the future time step. Fig. 15 shows a case when the CVC fails to capture the full shape of the object due to its vulnerability against occlusions. . As depicted, due to the occlusion, CVC fails to cover the whole extent of the object. Note that we have visualized both the Lidar points from the current frame (in gray) as well as the aggregated points (in green) which are used to represent the true object shape.